Comparison of Models: Quality, Performance & Price Analysis

Comparison and analysis of AI models across key performance metrics including quality, price, output speed, latency, context window & others. Click on any model to see detailed metrics. For more details including relating to our methodology, see our FAQs.

Model Comparison Summary

Quality:o1-preview logo o1-preview and o1-mini logo o1-mini are the highest quality models, followed by Claude 3.5 Sonnet (Oct) logo Claude 3.5 Sonnet (Oct) & Gemini 1.5 Pro (Sep) logo Gemini 1.5 Pro (Sep).Output Speed (tokens/s):Llama 3.2 1B logo Llama 3.2 1B (552 t/s) and Gemini 1.5 Flash (May) logo Gemini 1.5 Flash (May) (313 t/s) are the fastest models, followed by Gemini 1.5 Flash-8B logo Gemini 1.5 Flash-8B & Gemini 1.5 Flash (Sep) logo Gemini 1.5 Flash (Sep).Latency (seconds):Aya Expanse 32B logo Aya Expanse 32B (0.21s) and  Aya Expanse 8B logo Aya Expanse 8B (0.25s) are the lowest latency models, followed by Gemini 1.5 Flash (May) logo Gemini 1.5 Flash (May) & OpenChat 3.5 logo OpenChat 3.5.Price ($ per M tokens):Ministral 3B logo Ministral 3B ($0.04) and Llama 3.2 1B logo Llama 3.2 1B ($0.05) are the cheapest models, followed by OpenChat 3.5 logo OpenChat 3.5 & Gemini 1.5 Flash-8B logo Gemini 1.5 Flash-8B.Context Window:Gemini 1.5 Pro (Sep) logo Gemini 1.5 Pro (Sep) (2m) and Gemini 1.5 Pro (May) logo Gemini 1.5 Pro (May) (2m) are the largest context window models, followed by Gemini 1.5 Flash-8B logo Gemini 1.5 Flash-8B & Gemini 1.5 Flash (Sep) logo Gemini 1.5 Flash (Sep).

Highlights

Quality
Artificial Analysis Quality Index; Higher is better
Speed
Output Tokens per Second; Higher is better
Price
USD per 1M Tokens; Lower is better
Parallel Queries:
Prompt Length:
Further details
Model NameFurther analysis
OpenAI logo
OpenAI logoo1-preview
OpenAI logoo1-mini
OpenAI logoGPT-4o (Aug '24)
OpenAI logoGPT-4o (May '24)
OpenAI logoGPT-4o mini
OpenAI logoGPT-4 Turbo
OpenAI logoGPT-3.5 Turbo
OpenAI logoGPT-4
OpenAI logoGPT-3.5 Turbo Instruct
Meta logo
Meta logoLlama 3.1 Instruct 405B
Meta logoLlama 3.2 Instruct 90B (Vision)
Meta logoLlama 3.1 Instruct 70B
Meta logoLlama 3.2 Instruct 11B (Vision)
Meta logoLlama 3.1 Instruct 8B
Meta logoLlama 3.2 Instruct 3B
Meta logoLlama 3.2 Instruct 1B
Meta logoLlama 3 Instruct 70B
Meta logoLlama 3 Instruct 8B
Meta logoLlama 2 Chat 70B
Meta logoLlama 2 Chat 13B
Meta logoLlama 2 Chat 7B
Google logo
Google logoGemini 1.5 Pro (Sep '24)
Google logoGemini 1.5 Flash (Sep '24)
Google logoGemma 2 27B
Google logoGemma 2 9B
Google logoGemini 1.5 Pro (May '24)
Google logoGemini 1.5 Flash (May '24)
Google logoGemini 1.5 Flash-8B
Google logoGemini 1.0 Pro
Anthropic logo
Anthropic logoClaude 3.5 Sonnet (Oct '24)
Anthropic logoClaude 3.5 Sonnet (June '24)
Anthropic logoClaude 3 Opus
Anthropic logoClaude 3.5 Haiku
Anthropic logoClaude 3 Haiku
Anthropic logoClaude 3 Sonnet
Mistral logo
Mistral logoMistral Large 2
Mistral logoMixtral 8x22B Instruct
Mistral logoMistral Small (Sep '24)
Mistral logoPixtral 12B (2409)
Mistral logoMinistral 8B
Mistral logoMistral NeMo
Mistral logoMinistral 3B
Mistral logoMixtral 8x7B Instruct
Mistral logoCodestral-Mamba
Mistral logoMistral Large
Mistral logoMistral Small (Feb '24)
Mistral logoMistral 7B Instruct
Mistral logoMistral Medium
Mistral logoCodestral
Cohere logo
Cohere logoCommand-R+ (Aug '24)
Cohere logoCommand-R (Aug '24)
Cohere logoCommand-R+ (Apr '24)
Cohere logoCommand-R (Mar '24)
Cohere logoAya Expanse 32B
Cohere logoAya Expanse 8B
Perplexity logo
Perplexity logoSonar 3.1 Large
Perplexity logoSonar 3.1 Small
xAI logo
xAI logoGrok Beta
Microsoft Azure logo
Microsoft Azure logoPhi-3 Medium Instruct 14B
Upstage logo
Upstage logoSolar Pro
Upstage logoSolar Mini
Databricks logo
Databricks logoDBRX Instruct
NVIDIA logo
NVIDIA logoLlama 3.1 Nemotron Instruct 70B
Reka AI logo
Reka AI logoReka Flash (Sep '24)
Reka AI logoReka Core
Reka AI logoReka Flash (Feb '24)
Reka AI logoReka Edge
AI21 Labs logo
AI21 Labs logoJamba 1.5 Large
AI21 Labs logoJamba 1.5 Mini
AI21 Labs logoJamba Instruct
DeepSeek logo
DeepSeek logoDeepSeek-Coder-V2
DeepSeek logoDeepSeek-V2-Chat
DeepSeek logoDeepSeek-V2.5
Alibaba logo
Alibaba logoQwen2.5 Instruct 72B
Alibaba logoQwen2 Instruct 72B
01.AI logo
01.AI logoYi-Large
OpenChat logo
OpenChat logoOpenChat 3.5 (1210)

Models compared: OpenAI: GPT 4o Audio, GPT 4o Realtime, GPT 4o Speech Pipeline, GPT-3.5 Turbo, GPT-3.5 Turbo (0125), GPT-3.5 Turbo (1106), GPT-3.5 Turbo Instruct, GPT-4, GPT-4 Turbo, GPT-4 Turbo (0125), GPT-4 Vision, GPT-4o, GPT-4o (May '24), GPT-4o mini, o1-mini, and o1-preview, Meta: Code Llama 70B, Llama 2 Chat 13B, Llama 2 Chat 70B, Llama 2 Chat 7B, Llama 3 70B, Llama 3 8B, Llama 3.1 405B, Llama 3.1 70B, Llama 3.1 8B, Llama 3.2 11B (Vision), Llama 3.2 1B, Llama 3.2 3B, and Llama 3.2 90B (Vision), Google: Gemini 1.0 Pro, Gemini 1.5 Flash (May), Gemini 1.5 Flash (Sep), Gemini 1.5 Flash-8B, Gemini 1.5 Pro (May), Gemini 1.5 Pro (Sep), Gemma 2 27B, Gemma 2 9B, and Gemma 7B, Anthropic: Claude 2.0, Claude 2.1, Claude 3 Haiku, Claude 3 Opus, Claude 3 Sonnet, Claude 3.5 Haiku, Claude 3.5 Sonnet (June), Claude 3.5 Sonnet (Oct), and Claude Instant, Mistral: Codestral, Codestral-Mamba, Ministral 3B, Ministral 8B, Mistral 7B, Mistral Large, Mistral Large 2, Mistral Medium, Mistral NeMo, Mistral Small (Feb '24), Mistral Small (Sep '24), Mixtral 8x22B, Mixtral 8x7B, and Pixtral 12B, Cohere: Aya Expanse 32B, Aya Expanse 8B, Command, Command Light, Command-R, Command-R (Mar '24), Command-R+ (Apr '24), and Command-R+, Perplexity: PPLX-70B Online, PPLX-7B-Online, Sonar 3.1 Large, Sonar 3.1 Small , Sonar Large, and Sonar Small, xAI: Grok Beta and Grok-1, OpenChat: OpenChat 3.5, Microsoft Azure: Phi-3 Medium 14B and Phi-3 Mini, Upstage: Solar Mini and Solar Pro, Databricks: DBRX, NVIDIA: Llama 3.1 Nemotron 70B, IBM: Granite 3.0 2B, OpenVoice: Granite 3.0 8B, Reka AI: Reka Core, Reka Edge, Reka Flash (Feb '24), and Reka Flash, Other: LLaVA-v1.5-7B, AI21 Labs: Jamba 1.5 Large, Jamba 1.5 Mini, and Jamba Instruct, DeepSeek: DeepSeek-Coder-V2, DeepSeek-V2, and DeepSeek-V2.5, Snowflake: Arctic, Alibaba: Qwen2 72B and Qwen2.5 72B, and 01.AI: Yi-Large.