LLM Leaderboard - Comparison of GPT-4o, Llama 3, Mistral, Gemini and over 30 models
Comparison and ranking the performance of over 30 AI models (LLMs) across key metrics including quality, price, performance and speed (output speed - tokens per second & latency - TTFT), context window & others. For more details including relating to our methodology, see our FAQs.
HIGHLIGHTS








Features | Intelligence | Price | Output tokens/s | Latency | |||
---|---|---|---|---|---|---|---|
Further Analysis | |||||||
o3-mini | 200k | 63 | $1.93 | 145.1 | 16.33 | ||
o1 | 200k | 62 | $26.25 | ||||
DeepSeek R1 | ![]() | 128k | 60 | $0.96 | 27.3 | 60.46 | |
o1-mini | 128k | 54 | $1.93 | 186.7 | 10.55 | ||
DeepSeek R1 Distill Qwen 14B | ![]() | 128k | 50 | $0.88 | 104.0 | 14.50 | |
Gemini 2.0 Pro Experimental | 2m | 49 | $0.00 | 123.5 | 0.56 | ||
Gemini 2.0 Flash | 1m | 48 | $0.17 | 155.9 | 0.28 | ||
DeepSeek V3 | ![]() | 128k | 46 | $0.48 | 25.9 | 4.54 | |
DeepSeek R1 Distill Llama 70B | ![]() | 128k | 45 | $0.81 | 123.2 | 12.69 | |
Qwen2.5 Max | 32k | 45 | $2.80 | 36.1 | 1.29 | ||
DeepSeek R1 Distill Qwen 32B | ![]() | 128k | 45 | $0.30 | 38.7 | 14.73 | |
Gemini 1.5 Pro (Sep) | 2m | 45 | $2.19 | 0.0 | 0.10 | ||
Claude 3.5 Sonnet (Oct) | 200k | 44 | $6.00 | 72.5 | 1.58 | ||
QwQ 32B-Preview | 33k | 43 | $0.58 | 65.8 | 0.58 | ||
Gemini 2.0 Flash-Lite (Preview) | 1m | 42 | $0.13 | 257.8 | 0.26 | ||
GPT-4o (Nov '24) | 128k | 41 | $4.38 | 55.1 | 0.58 | ||
Llama 3.3 70B | 128k | 41 | $0.62 | 96.3 | 0.52 | ||
GPT-4o (ChatGPT) | 128k | 41 | $7.50 | 101.7 | 0.48 | ||
GPT-4o (Aug '24) | 128k | 41 | $4.38 | 70.9 | 0.40 | ||
GPT-4o (May '24) | 128k | 41 | $7.50 | 68.6 | 0.41 | ||
Llama 3.1 405B | 128k | 40 | $3.50 | 24.1 | 0.73 | ||
Qwen2.5 72B | 131k | 40 | $0.00 | 18.0 | 1.16 | ||
Phi-4 | ![]() | 16k | 40 | $0.12 | 37.9 | 0.26 | |
Tulu3 405B | 128k | 40 | $6.25 | 178.9 | 0.68 | ||
MiniMax-Text-01 | ![]() | 4m | 40 | $0.42 | 42.4 | 0.79 | |
Mistral Large 2 (Nov '24) | ![]() | 128k | 38 | $3.00 | 46.3 | 0.34 | |
Grok Beta | 128k | 38 | $7.50 | 67.6 | 0.33 | ||
Pixtral Large | ![]() | 128k | 37 | $3.00 | 35.4 | 0.40 | |
Qwen2.5 Instruct 32B | 128k | 37 | $0.79 | ||||
Llama 3.1 Nemotron 70B | 128k | 37 | $0.27 | 47.1 | 0.60 | ||
Nova Pro | ![]() | 300k | 37 | $1.40 | 88.7 | 0.40 | |
Mistral Large 2 (Jul '24) | ![]() | 128k | 37 | $3.00 | 30.6 | 0.37 | |
Qwen2.5 Coder 32B | 131k | 36 | $0.80 | 91.8 | 0.36 | ||
GPT-4o mini | 128k | 36 | $0.26 | 79.2 | 0.41 | ||
DeepSeek R1 Distill Llama 8B | ![]() | 128k | 35 | $0.04 | 47.8 | 9.57 | |
Llama 3.1 70B | 128k | 35 | $0.72 | 73.0 | 0.51 | ||
Mistral Small 3 | ![]() | 32k | 35 | $0.15 | 145.8 | 0.31 | |
Claude 3 Opus | 200k | 35 | $30.00 | 27.8 | 0.98 | ||
Claude 3.5 Haiku | 200k | 35 | $1.60 | 65.6 | 0.57 | ||
Gemini 1.5 Pro (May) | 2m | 34 | $2.19 | 0.0 | 0.10 | ||
Qwen Turbo | 1m | 34 | $0.09 | 85.7 | 1.12 | ||
Llama 3.2 90B (Vision) | 128k | 33 | $0.90 | 45.7 | 0.31 | ||
Mistral Saba | ![]() | 32k | 32 | $0.30 | 94.2 | 0.31 | |
Jamba 1.5 Large | 256k | 29 | $3.50 | 0.0 | 0.00 | ||
Gemini 1.5 Flash (May) | 1m | 28 | $0.13 | 0.0 | 0.11 | ||
Nova Micro | ![]() | 130k | 28 | $0.06 | 194.6 | 0.35 | |
Yi-Large | ![]() | 32k | 28 | $3.00 | 68.7 | 0.43 | |
Codestral (Jan '25) | ![]() | 256k | 28 | $0.45 | 211.3 | 0.27 | |
Llama 3 70B | 8k | 27 | $0.88 | 47.0 | 0.40 | ||
Mistral Small (Sep '24) | ![]() | 33k | 27 | $0.30 | 84.3 | 0.32 | |
Mistral Large (Feb '24) | ![]() | 33k | 26 | $6.00 | 36.4 | 0.40 | |
Mixtral 8x22B | ![]() | 65k | 26 | $3.00 | 75.4 | 0.28 | |
Phi-3 Medium 14B | ![]() | 128k | 25 | $0.30 | 52.4 | 0.43 | |
Mistral Medium | ![]() | 33k | 24 | $4.09 | 43.7 | 0.36 | |
Llama 3.1 8B | 128k | 24 | $0.10 | 185.9 | 0.35 | ||
Pixtral 12B | ![]() | 128k | 23 | $0.15 | 102.5 | 0.29 | |
Mistral Small (Feb '24) | ![]() | 33k | 23 | $1.50 | 149.5 | 0.27 | |
Ministral 8B | ![]() | 128k | 22 | $0.10 | 140.7 | 0.33 | |
Llama 3.2 11B (Vision) | 128k | 22 | $0.18 | 143.7 | 0.31 | ||
Command-R+ | 128k | 21 | $4.38 | 73.7 | 0.23 | ||
Codestral (May '24) | ![]() | 33k | 20 | $0.30 | 84.4 | 0.28 | |
Aya Expanse 32B | 128k | 20 | $0.75 | 120.9 | 0.16 | ||
DBRX | 33k | 20 | $1.13 | 66.5 | 0.55 | ||
Ministral 3B | ![]() | 128k | 20 | $0.04 | 220.1 | 0.30 | |
Mistral NeMo | ![]() | 128k | 20 | $0.15 | 118.2 | 0.26 | |
DeepSeek R1 Distill Qwen 1.5B | ![]() | 128k | 20 | $0.18 | 368.9 | 6.39 | |
OpenChat 3.5 | ![]() | 8k | 16 | $0.06 | 80.9 | 0.30 | |
Command-R (Mar '24) | 128k | 15 | $0.75 | 170.8 | 0.17 | ||
Codestral-Mamba | ![]() | 256k | 14 | $0.25 | 94.8 | 0.45 | |
o1-preview | 128k | $26.25 | 113.7 | 32.03 | |||
Grok 3 | 128k | $0.00 | |||||
Grok 3 mini | 128k | $0.00 | |||||
Grok 3 Reasoning Beta | 128k | $0.00 | |||||
Grok 3 mini Reasoning | 128k | $0.00 | |||||
o3 | 128k | $0.00 | |||||
Llama 3.2 3B | 128k | $0.06 | 126.8 | 0.47 | |||
Llama 3.2 1B | 128k | $0.04 | 186.8 | 0.35 | |||
Gemini 2.0 Flash (exp) | 1m | $0.00 | 156.9 | 0.26 | |||
Gemini 1.5 Flash (Sep) | 1m | $0.13 | 0.0 | 0.10 | |||
Gemma 2 27B | 8k | $0.26 | |||||
Gemma 2 9B | 8k | $0.12 | |||||
Gemini 1.5 Flash-8B | 1m | $0.07 | |||||
Gemini Experimental (Nov) | 2m | $0.00 | |||||
Claude 3.5 Sonnet (June) | 200k | $6.00 | 85.5 | 1.02 | |||
Claude 3 Haiku | 200k | $0.50 | 136.5 | 0.54 | |||
Mixtral 8x7B | ![]() | 33k | $0.70 | 102.5 | 0.30 | ||
DeepSeek-V2.5 (Dec '24) | ![]() | 128k | $0.17 | ||||
DeepSeek-Coder-V2 | ![]() | 128k | $0.17 | ||||
DeepSeek LLM 67B (V1) | ![]() | 4k | $0.90 | ||||
DeepSeek-V2.5 | ![]() | 128k | $1.09 | ||||
DeepSeek-V2 | ![]() | 128k | $0.17 | ||||
Sonar | 127k | $1.00 | |||||
Sonar Pro | 200k | $6.00 | |||||
Sonar Reasoning | 127k | $2.00 | |||||
Nova Lite | ![]() | 300k | $0.10 | 140.9 | 0.37 | ||
Solar Mini | ![]() | 4k | $0.15 | ||||
Reka Flash | ![]() | 128k | $0.35 | ||||
Reka Core | ![]() | 128k | $2.00 | ||||
Reka Flash (Feb '24) | ![]() | 128k | $0.35 | ||||
Reka Edge | ![]() | 128k | $0.10 | ||||
Command-R+ (Apr '24) | 128k | $6.00 | 76.2 | 0.26 | |||
Aya Expanse 8B | 8k | $0.75 | 166.7 | 0.14 | |||
Command-R | 128k | $0.26 | 94.0 | 0.17 | |||
Jamba 1.5 Mini | 256k | $0.25 | 0.0 | 0.00 | |||
Qwen2 72B | 131k | $0.00 | |||||
Qwen Chat 72B | 34k | $1.00 | |||||
Qwen1.5 Chat 110B | 32k | $0.00 | |||||
GPT-4 Turbo | 128k | $15.00 | 32.6 | 1.11 | |||
GPT-4 | 8k | $37.50 | |||||
Llama 3 8B | 8k | $0.10 | 103.5 | 0.36 | |||
Llama 2 Chat 7B | 4k | $0.10 | 123.6 | 0.42 | |||
Gemini 1.0 Pro | 33k | $0.75 | 103.6 | 1.24 | |||
Claude 3 Sonnet | 200k | $6.00 | 58.1 | 0.58 | |||
Claude 2.1 | 200k | $12.00 | 13.7 | 0.82 | |||
Claude 2.0 | 100k | $12.00 | 29.5 | 0.81 | |||
Mistral 7B | ![]() | 8k | $0.25 | 128.7 | 0.26 | ||
Sonar 3.1 Small | 127k | $0.20 | |||||
Sonar 3.1 Large | 127k | $1.00 | |||||
Jamba Instruct | 256k | $0.55 | 0.0 | 0.00 |
Key definitions
Models compared: OpenAI: GPT 4o Audio, GPT 4o Realtime, GPT 4o Speech Pipeline, GPT-3.5 Turbo, GPT-3.5 Turbo (0125), GPT-3.5 Turbo (0314), GPT-3.5 Turbo (1106), GPT-3.5 Turbo Instruct, GPT-4, GPT-4 Turbo, GPT-4 Turbo (0125), GPT-4 Turbo (1106), GPT-4 Vision, GPT-4o (Aug '24), GPT-4o (ChatGPT), GPT-4o (May '24), GPT-4o (Nov '24), GPT-4o Realtime (Dec '24), GPT-4o mini, GPT-4o mini Realtime (Dec '24), o1, o1-mini, o1-preview, o3, o3-mini, and o3-mini (high), Meta: Code Llama 70B, Llama 2 Chat 13B, Llama 2 Chat 70B, Llama 2 Chat 7B, Llama 3 70B, Llama 3 8B, Llama 3.1 405B, Llama 3.1 70B, Llama 3.1 8B, Llama 3.2 11B (Vision), Llama 3.2 1B, Llama 3.2 3B, Llama 3.2 90B (Vision), and Llama 3.3 70B, Google: Gemini 1.0 Pro, Gemini 1.5 Flash (May), Gemini 1.5 Flash (Sep), Gemini 1.5 Flash-8B, Gemini 1.5 Pro (May), Gemini 1.5 Pro (Sep), Gemini 2.0 Flash, Gemini 2.0 Flash (exp), Gemini 2.0 Flash Thinking exp. (Dec '24), Gemini 2.0 Flash Thinking exp. (Jan '25), Gemini 2.0 Flash-Lite (Preview), Gemini 2.0 Pro Experimental, Gemini Experimental (Nov), Gemma 2 27B, Gemma 2 9B, and Gemma 7B, Anthropic: Claude 2.0, Claude 2.1, Claude 3 Haiku, Claude 3 Opus, Claude 3 Sonnet, Claude 3.5 Haiku, Claude 3.5 Sonnet (June), Claude 3.5 Sonnet (Oct), and Claude Instant, Mistral: Codestral (Jan '25), Codestral (May '24), Codestral-Mamba, Ministral 3B, Ministral 8B, Mistral 7B, Mistral Large (Feb '24), Mistral Large 2 (Jul '24), Mistral Large 2 (Nov '24), Mistral Medium, Mistral NeMo, Mistral Saba, Mistral Small (Feb '24), Mistral Small (Sep '24), Mistral Small 3, Mixtral 8x22B, Mixtral 8x7B, Pixtral 12B, and Pixtral Large, DeepSeek: DeepSeek LLM 67B (V1), DeepSeek R1, DeepSeek R1 Distill Llama 70B, DeepSeek R1 Distill Llama 8B, DeepSeek R1 Distill Qwen 1.5B, DeepSeek R1 Distill Qwen 14B, DeepSeek R1 Distill Qwen 32B, DeepSeek V3, DeepSeek-Coder-V2, DeepSeek-V2, DeepSeek-V2.5, DeepSeek-V2.5 (Dec '24), DeepSeek-VL2, and Janus Pro 7B, Perplexity: PPLX-70B Online, PPLX-7B-Online, Sonar, Sonar 3.1 Huge, Sonar 3.1 Large, Sonar 3.1 Small , Sonar Large, Sonar Pro, Sonar Reasoning, Sonar Reasoning Pro, and Sonar Small, xAI: Grok 2, Grok 3, Grok 3 Reasoning Beta, Grok 3 mini, Grok 3 mini Reasoning, Grok Beta, and Grok-1, OpenChat: OpenChat 3.5, Amazon: Nova Lite, Nova Micro, and Nova Pro, Microsoft Azure: Phi-3 Medium 14B, Phi-3 Mini, and Phi-4, Upstage: Solar Mini, Solar Pro, and Solar Pro (Nov '24), Databricks: DBRX, MiniMax: MiniMax-Text-01, NVIDIA: Cosmos Nemotron 34B and Llama 3.1 Nemotron 70B, StepFun: step-2-16k, step-1-128k, step-1-256k, step-1-32k, step-1-8k, step-1-flash, step-2-16k-exp, and step-2-mini, IBM: Granite 3.0 2B, OpenVoice: Granite 3.0 8B, Reka AI: Reka Core, Reka Edge, Reka Flash (Feb '24), and Reka Flash, Other: LLaVA-v1.5-7B, Cohere: Aya Expanse 32B, Aya Expanse 8B, Command, Command Light, Command R7B, Command-R, Command-R (Mar '24), Command-R+ (Apr '24), and Command-R+, AI21 Labs: Jamba 1.5 Large, Jamba 1.5 Mini, and Jamba Instruct, Snowflake: Arctic, Alibaba: QwQ 32B-Preview, Qwen Chat 72B, Qwen Plus, Qwen Turbo, Qwen1.5 Chat 110B, Qwen1.5 Chat 14B, Qwen1.5 Chat 32B, Qwen1.5 Chat 72B, Qwen1.5 Chat 7B, Qwen2 72B, Qwen2 Instruct 7B, Qwen2 Instruct A14B 57B, Qwen2-VL 72B, Qwen2.5 Coder 32B, Qwen2.5 Instruct 14B, Qwen2.5 Instruct 32B, Qwen2.5 72B, Qwen2.5 Instruct 7B, Qwen2.5 Max, and Qwen2.5 Max 01-29, and 01.AI: Yi-Large.