Comparison of Models: Quality, Performance & Price Analysis

Comparison and analysis of AI models across key performance metrics including quality, price, output speed, latency, context window & others. Click on any model to see detailed metrics. For more details including relating to our methodology, see our FAQs.

Model Comparison Summary

Quality:o1-preview logo o1-preview and o1-mini logo o1-mini are the highest quality models, followed by Gemini 1.5 Pro (Sep '24) logo Gemini 1.5 Pro (Sep '24) & GPT-4o logo GPT-4o.Output Speed (tokens/s):Llama 3.2 1B logo Llama 3.2 1B (556 t/s) and Gemini 1.5 Flash (May '24) logo Gemini 1.5 Flash (May '24) (306 t/s) are the fastest models, followed by Gemini 1.5 Flash-8B logo Gemini 1.5 Flash-8B & Gemini 1.5 Flash (Sep '24) logo Gemini 1.5 Flash (Sep '24).Latency (seconds):Gemma 2 9B logo Gemma 2 9B (0.29s) and  Sonar 3.1 Small  logo Sonar 3.1 Small (0.31s) are the lowest latency models, followed by OpenChat 3.5 logo OpenChat 3.5 & Sonar 3.1 Large logo Sonar 3.1 Large.Price ($ per M tokens):Llama 3.2 1B logo Llama 3.2 1B ($0.05) and OpenChat 3.5 logo OpenChat 3.5 ($0.06) are the cheapest models, followed by Gemini 1.5 Flash-8B logo Gemini 1.5 Flash-8B & Llama 3.2 3B logo Llama 3.2 3B.Context Window:Gemini 1.5 Pro (Sep '24) logo Gemini 1.5 Pro (Sep '24) (2m) and Gemini 1.5 Pro (May '24) logo Gemini 1.5 Pro (May '24) (2m) are the largest context window models, followed by Gemini 1.5 Flash-8B logo Gemini 1.5 Flash-8B & Gemini 1.5 Flash (Sep '24) logo Gemini 1.5 Flash (Sep '24).

Highlights

Quality
Artificial Analysis Quality Index; Higher is better
Speed
Output Tokens per Second; Higher is better
Price
USD per 1M Tokens; Lower is better
Parallel Queries:
Prompt Length:
Further details
Model NameFurther analysis
OpenAI logo
OpenAI logoo1-preview
OpenAI logoo1-mini
OpenAI logoGPT-4o (Aug '24)
OpenAI logoGPT-4o (May '24)
OpenAI logoGPT-4o mini
OpenAI logoGPT-4 Turbo
OpenAI logoGPT-3.5 Turbo
OpenAI logoGPT-3.5 Turbo Instruct
OpenAI logoGPT-4
Meta logo
Meta logoLlama 3.1 Instruct 405B
Meta logoLlama 3.2 Instruct 90B (Vision)
Meta logoLlama 3.1 Instruct 70B
Meta logoLlama 3.2 Instruct 11B (Vision)
Meta logoLlama 3.1 Instruct 8B
Meta logoLlama 3.2 Instruct 3B
Meta logoLlama 3.2 Instruct 1B
Meta logoLlama 3 Instruct 70B
Meta logoLlama 3 Instruct 8B
Meta logoLlama 2 Chat 70B
Meta logoLlama 2 Chat 13B
Meta logoLlama 2 Chat 7B
Google logo
Google logoGemini 1.5 Pro (Sep '24)
Google logoGemini 1.5 Flash-8B
Google logoGemini 1.5 Flash (Sep '24)
Google logoGemma 2 27B
Google logoGemma 2 9B
Google logoGemini 1.5 Flash (May '24)
Google logoGemini 1.5 Pro (May '24)
Google logoGemini 1.0 Pro
Anthropic logo
Anthropic logoClaude 3.5 Sonnet
Anthropic logoClaude 3 Opus
Anthropic logoClaude 3 Haiku
Anthropic logoClaude 3 Sonnet
Mistral logo
Mistral logoMistral Large 2
Mistral logoMixtral 8x22B Instruct
Mistral logoMistral Small (Sep '24)
Mistral logoPixtral 12B (2409)
Mistral logoMistral NeMo
Mistral logoMixtral 8x7B Instruct
Mistral logoCodestral-Mamba
Mistral logoMistral Large
Mistral logoMistral Small (Feb '24)
Mistral logoMistral 7B Instruct
Mistral logoCodestral
Mistral logoMistral Medium
Cohere logo
Cohere logoCommand-R+ (Aug '24)
Cohere logoCommand-R (Aug '24)
Cohere logoCommand-R+ (Apr '24)
Cohere logoCommand-R (Mar '24)
Perplexity logo
Perplexity logoSonar 3.1 Small
Perplexity logoSonar 3.1 Large
Microsoft Azure logo
Microsoft Azure logoPhi-3 Medium Instruct 14B
Upstage logo
Upstage logoSolar Mini
Upstage logoSolar Pro
Databricks logo
Databricks logoDBRX Instruct
Reka AI logo
Reka AI logoReka Core
Reka AI logoReka Flash
Reka AI logoReka Edge
AI21 Labs logo
AI21 Labs logoJamba 1.5 Large
AI21 Labs logoJamba 1.5 Mini
AI21 Labs logoJamba Instruct
DeepSeek logo
DeepSeek logoDeepSeek-Coder-V2
DeepSeek logoDeepSeek-V2-Chat
DeepSeek logoDeepSeek-V2.5
Alibaba logo
Alibaba logoQwen2.5 Instruct 72B
Alibaba logoQwen2 Instruct 72B
01.AI logo
01.AI logoYi-Large
OpenChat logo
OpenChat logoOpenChat 3.5 (1210)

Models compared: OpenAI: GPT-3.5 Turbo, GPT-3.5 Turbo (0125), GPT-3.5 Turbo (1106), GPT-3.5 Turbo Instruct, GPT-4, GPT-4 Turbo, GPT-4 Turbo (0125), GPT-4 Vision, GPT-4o, GPT-4o (May '24), GPT-4o mini, o1-mini, and o1-preview, Meta: Code Llama 70B, Llama 2 Chat 13B, Llama 2 Chat 70B, Llama 2 Chat 7B, Llama 3 70B, Llama 3 8B, Llama 3.1 405B, Llama 3.1 70B, Llama 3.1 8B, Llama 3.2 11B (Vision), Llama 3.2 1B, Llama 3.2 3B, and Llama 3.2 90B (Vision), Google: Gemini 1.0 Pro, Gemini 1.5 Flash (May '24), Gemini 1.5 Flash (Sep '24), Gemini 1.5 Flash-8B, Gemini 1.5 Pro (May '24), Gemini 1.5 Pro (Sep '24), Gemma 2 27B, Gemma 2 9B, and Gemma 7B, Anthropic: Claude 2.0, Claude 2.1, Claude 3 Haiku, Claude 3 Opus, Claude 3 Sonnet, Claude 3.5 Sonnet, and Claude Instant, Mistral: Codestral, Codestral-Mamba, Mistral 7B, Mistral Large, Mistral Large 2, Mistral Medium, Mistral NeMo, Mistral Small (Feb '24), Mistral Small (Sep '24), Mixtral 8x22B, Mixtral 8x7B, and Pixtral 12B, Cohere: Command, Command Light, Command-R, Command-R (Mar '24), Command-R+ (Apr '24), and Command-R+, Perplexity: PPLX-70B Online, PPLX-7B-Online, Sonar 3.1 Large, Sonar 3.1 Small , Sonar Large, and Sonar Small, xAI: Grok-1, OpenChat: OpenChat 3.5, Microsoft Azure: Phi-3 Medium 14B and Phi-3 Mini, Upstage: Solar Mini and Solar Pro, Databricks: DBRX, Reka AI: Reka Core, Reka Edge, and Reka Flash, Other: LLaVA-v1.5-7B, AI21 Labs: Jamba 1.5 Large, Jamba 1.5 Mini, and Jamba Instruct, DeepSeek: DeepSeek-Coder-V2, DeepSeek-V2, and DeepSeek-V2.5, Snowflake: Arctic, Alibaba: Qwen2 72B and Qwen2.5 72B, and 01.AI: Yi-Large.