Gemini Experimental (Dec '24): Quality, Performance & Price Analysis

Analysis of Google's Gemini Experimental (Dec '24) and comparison to other AI models across key metrics including quality, price, performance (tokens per second & time to first token), context window & more. Click on any model to compare API providers for that model. For more details including relating to our methodology, see our FAQs.

For analysis of API providers see

Creator:

Google

License:

Proprietary

Context window:

33k

Link:

Visit

Comparison Summary

Quality:Price:Speed:

Gemini Experimental (Nov) is slower compared to average, with a output speed of 40.4 tokens per second.

Latency:

Gemini Experimental (Nov) has a higher latency compared to average, taking 2.62s to receive the first token (TTFT).

Context Window:

Gemini Experimental (Nov) has a smaller context windows than average, with a context window of 33k tokens.

Highlights

Quality

Artificial Analysis Quality Index; Higher is better

Speed

Output Tokens per Second; Higher is better

Price

USD per 1M Tokens; Lower is better

Model Name	Creator	License	Context Window

o1-preview	OpenAI	Proprietary	128k
o1-mini	OpenAI	Proprietary	128k
GPT-4o (Aug '24)	OpenAI	Proprietary	128k
GPT-4o (May '24)	OpenAI	Proprietary	128k
GPT-4o (Nov '24)	OpenAI	Proprietary	128k
GPT-4o mini	OpenAI	Proprietary	128k
GPT-4 Turbo	OpenAI	Proprietary	128k
GPT-3.5 Turbo	OpenAI	Proprietary	16k
GPT-4	OpenAI	Proprietary	8k

Llama 3.3 Instruct 70B	Meta	Open	128k
Llama 3.1 Instruct 405B	Meta	Open	128k
Llama 3.1 Instruct 70B	Meta	Open	128k
Llama 3.2 Instruct 90B (Vision)	Meta	Open	128k
Llama 3.2 Instruct 11B (Vision)	Meta	Open	128k
Llama 3.1 Instruct 8B	Meta	Open	128k
Llama 3.2 Instruct 3B	Meta	Open	128k
Llama 3.2 Instruct 1B	Meta	Open	128k
Llama 3 Instruct 70B	Meta	Open	8k
Llama 3 Instruct 8B	Meta	Open	8k
Llama 2 Chat 13B	Meta	Open	4k
Llama 2 Chat 7B	Meta	Open	4k

Gemini 1.5 Pro (Sep '24)	Google	Proprietary	2m
Gemini 1.5 Flash (Sep '24)	Google	Proprietary	1m
Gemma 2 27B	Google	Open	8k
Gemma 2 9B	Google	Open	8k
Gemini Experimental (Dec '24)	Google	Proprietary	33k
Gemini 1.5 Pro (May '24)	Google	Proprietary	2m
Gemini 1.5 Flash (May '24)	Google	Proprietary	1m
Gemini 1.5 Flash-8B	Google	Proprietary	1m
Gemini 1.0 Pro	Google	Proprietary	33k

Claude 3.5 Sonnet (Oct '24)	Anthropic	Proprietary	200k
Claude 3.5 Sonnet (June '24)	Anthropic	Proprietary	200k
Claude 3 Opus	Anthropic	Proprietary	200k
Claude 3.5 Haiku	Anthropic	Proprietary	200k
Claude 3 Haiku	Anthropic	Proprietary	200k
Claude 3 Sonnet	Anthropic	Proprietary	200k

Pixtral Large	Mistral	Open	128k
Mistral Large 2 (Jul '24)	Mistral	Open	128k
Mistral Large 2 (Nov '24)	Mistral	Open	128k
Mistral Small (Sep '24)	Mistral	Open	128k
Mixtral 8x22B Instruct	Mistral	Open	65k
Pixtral 12B (2409)	Mistral	Open	128k
Ministral 8B	Mistral	Open	128k
Mistral NeMo	Mistral	Open	128k
Ministral 3B	Mistral	Proprietary	128k
Mixtral 8x7B Instruct	Mistral	Open	33k
Codestral-Mamba	Mistral	Open	256k
Mistral Small (Feb '24)	Mistral	Proprietary	33k
Mistral Large (Feb '24)	Mistral	Proprietary	33k
Mistral 7B Instruct	Mistral	Open	33k
Codestral	Mistral	Open	33k
Mistral Medium	Mistral	Proprietary	33k

Command-R+ (Aug '24)	Cohere	Open	128k
Command-R+ (Apr '24)	Cohere	Open	128k
Command-R (Mar '24)	Cohere	Open	128k
Command-R (Aug '24)	Cohere	Open	128k
Aya Expanse 32B	Cohere	Open	128k
Aya Expanse 8B	Cohere	Open	8k

Sonar 3.1 Large	Perplexity	Proprietary	131k
Sonar 3.1 Small	Perplexity	Proprietary	131k

Grok Beta	xAI	Proprietary	8k

Nova Pro	Amazon	Proprietary	300k
Nova Lite	Amazon	Proprietary	300k
Nova Micro	Amazon	Proprietary	130k

Phi-3 Medium Instruct 14B	Microsoft Azure	Open	128k

Solar Mini	Upstage	Open	4k

DBRX Instruct	Databricks	Open	33k

Llama 3.1 Nemotron Instruct 70B	NVIDIA	Open	128k

Reka Flash (Sep '24)	Reka AI	Proprietary	128k
Reka Core	Reka AI	Proprietary	128k
Reka Flash (Feb '24)	Reka AI	Proprietary	128k
Reka Edge	Reka AI	Proprietary	64k

Jamba 1.5 Large	AI21 Labs	Open	256k
Jamba 1.5 Mini	AI21 Labs	Open	256k
Jamba Instruct	AI21 Labs	Open	256k

DeepSeek-Coder-V2	DeepSeek	Open	128k
DeepSeek-V2-Chat	DeepSeek	Open	128k
DeepSeek-V2.5	DeepSeek	Open	128k

Qwen2.5 Instruct 72B	Alibaba	Open	131k
Qwen2.5 Coder Instruct 32B	Alibaba	Open	131k
Qwen2 Instruct 72B	Alibaba	Open	128k

Yi-Large	01.AI	Proprietary	32k

OpenChat 3.5 (1210)	OpenChat	Open	8k

Gemini Experimental (Dec '24): Quality, Performance & Price Analysis

Highlights

Navigation

Quality & Context Window

Pricing

Performance Summary

Speed

Latency

Total Response Time