Menu

logo
Artificial Analysis
HOME

Comparison of Models: Quality, Performance & Price Analysis

Comparison and analysis of AI models across key performance metrics including quality, price, output speed, latency, context window & others. Click on any model to see detailed metrics. For more details including relating to our methodology, see our FAQs.

Model Comparison Summary

Quality:o1-preview logo o1-previewĀ andĀ o1-mini logo o1-miniĀ are the highest quality models, followed by Gemini 1.5 Pro (Sep) logo Gemini 1.5 Pro (Sep) & Claude 3.5 Sonnet (Oct) logo Claude 3.5 Sonnet (Oct).Output Speed (tokens/s):Llama 3.2 1B logo Llama 3.2 1B (552 t/s)Ā andĀ Gemini 1.5 Flash (May) logo Gemini 1.5 Flash (May) (303 t/s)Ā are the fastest models, followed by Gemini 1.5 Flash-8B logo Gemini 1.5 Flash-8B & o1-mini logo o1-mini.Latency (seconds):Aya Expanse 32B logo Aya Expanse 32B (0.23s)Ā and Ā Llama 3.2 11B (Vision) logo Llama 3.2 11B (Vision) (0.29s)Ā are the lowest latency models, followed by Gemini 1.5 Flash (May) logo Gemini 1.5 Flash (May) & Llama 3.2 3B logo Llama 3.2 3B.Price ($ per M tokens):Gemini Experimental (Nov) logo Gemini Experimental (Nov) ($0.00)Ā andĀ Ministral 3B logo Ministral 3B ($0.04)Ā are the cheapest models, followed by Llama 3.2 1B logo Llama 3.2 1B & OpenChat 3.5 logo OpenChat 3.5.Context Window:Gemini 1.5 Pro (Sep) logo Gemini 1.5 Pro (Sep) (2m)Ā andĀ Gemini 1.5 Pro (May) logo Gemini 1.5 Pro (May) (2m)Ā are the largest context window models, followed by Gemini 1.5 Flash-8B logo Gemini 1.5 Flash-8B & Gemini 1.5 Flash (Sep) logo Gemini 1.5 Flash (Sep).

Highlights

Quality
Artificial Analysis Quality Index; Higher is better
Speed
Output Tokens per Second; Higher is better
Price
USD per 1M Tokens; Lower is better
Parallel Queries:
Prompt Length:
Further details
Model NameFurther analysis
OpenAI logo
OpenAI logoo1-preview
OpenAI logoo1-mini
OpenAI logoGPT-4o (Aug '24)
OpenAI logoGPT-4o (May '24)
OpenAI logoGPT-4o (Nov '24)
OpenAI logoGPT-4o mini
OpenAI logoGPT-4 Turbo
OpenAI logoGPT-3.5 Turbo
OpenAI logoGPT-4
Meta logo
Meta logoLlama 3.3 Instruct 70B
Meta logoLlama 3.1 Instruct 405B
Meta logoLlama 3.1 Instruct 70B
Meta logoLlama 3.2 Instruct 90B (Vision)
Meta logoLlama 3.2 Instruct 11B (Vision)
Meta logoLlama 3.1 Instruct 8B
Meta logoLlama 3.2 Instruct 3B
Meta logoLlama 3.2 Instruct 1B
Meta logoLlama 3 Instruct 70B
Meta logoLlama 3 Instruct 8B
Meta logoLlama 2 Chat 13B
Meta logoLlama 2 Chat 7B
Google logo
Google logoGemini 1.5 Pro (Sep '24)
Google logoGemini 1.5 Flash (Sep '24)
Google logoGemma 2 27B
Google logoGemma 2 9B
Google logoGemini 1.5 Flash-8B
Google logoGemini Experimental (Dec '24)
Google logoGemini 1.5 Flash (May '24)
Google logoGemini 1.5 Pro (May '24)
Google logoGemini 1.0 Pro
Anthropic logo
Anthropic logoClaude 3.5 Sonnet (Oct '24)
Anthropic logoClaude 3.5 Sonnet (June '24)
Anthropic logoClaude 3 Opus
Anthropic logoClaude 3.5 Haiku
Anthropic logoClaude 3 Haiku
Anthropic logoClaude 3 Sonnet
Mistral logo
Mistral logoPixtral Large
Mistral logoMistral Large 2 (Jul '24)
Mistral logoMistral Large 2 (Nov '24)
Mistral logoMistral Small (Sep '24)
Mistral logoMixtral 8x22B Instruct
Mistral logoPixtral 12B (2409)
Mistral logoMinistral 8B
Mistral logoMistral NeMo
Mistral logoMinistral 3B
Mistral logoMixtral 8x7B Instruct
Mistral logoCodestral-Mamba
Mistral logoMistral Small (Feb '24)
Mistral logoMistral Large (Feb '24)
Mistral logoMistral 7B Instruct
Mistral logoCodestral
Mistral logoMistral Medium
Cohere logo
Cohere logoCommand-R+ (Aug '24)
Cohere logoCommand-R+ (Apr '24)
Cohere logoCommand-R (Mar '24)
Cohere logoCommand-R (Aug '24)
Cohere logoAya Expanse 8B
Cohere logoAya Expanse 32B
Perplexity logo
Perplexity logoSonar 3.1 Small
Perplexity logoSonar 3.1 Large
xAI logo
xAI logoGrok Beta
Amazon logo
Amazon logoNova Pro
Amazon logoNova Lite
Amazon logoNova Micro
Microsoft Azure logo
Microsoft Azure logoPhi-3 Medium Instruct 14B
Upstage logo
Upstage logoSolar Mini
Databricks logo
Databricks logoDBRX Instruct
NVIDIA logo
NVIDIA logoLlama 3.1 Nemotron Instruct 70B
Reka AI logo
Reka AI logoReka Flash (Sep '24)
Reka AI logoReka Core
Reka AI logoReka Flash (Feb '24)
Reka AI logoReka Edge
AI21 Labs logo
AI21 Labs logoJamba 1.5 Mini
AI21 Labs logoJamba 1.5 Large
AI21 Labs logoJamba Instruct
DeepSeek logo
DeepSeek logoDeepSeek-Coder-V2
DeepSeek logoDeepSeek-V2.5
DeepSeek logoDeepSeek-V2-Chat
Alibaba logo
Alibaba logoQwen2.5 Instruct 72B
Alibaba logoQwen2.5 Coder Instruct 32B
Alibaba logoQwen2 Instruct 72B
01.AI logo
01.AI logoYi-Large
OpenChat logo
OpenChat logoOpenChat 3.5 (1210)

Models compared: OpenAI: GPT 4o Audio, GPT 4o Realtime, GPT 4o Speech Pipeline, GPT-3.5 Turbo, GPT-3.5 Turbo (0125), GPT-3.5 Turbo (1106), GPT-3.5 Turbo Instruct, GPT-4, GPT-4 Turbo, GPT-4 Turbo (0125), GPT-4 Vision, GPT-4o (Aug '24), GPT-4o (May '24), GPT-4o (Nov '24), GPT-4o mini, o1-mini, and o1-preview, Meta: Code Llama 70B, Llama 2 Chat 13B, Llama 2 Chat 70B, Llama 2 Chat 7B, Llama 3 70B, Llama 3 8B, Llama 3.1 405B, Llama 3.1 70B, Llama 3.1 8B, Llama 3.2 11B (Vision), Llama 3.2 1B, Llama 3.2 3B, Llama 3.2 90B (Vision), and Llama 3.3 70B, Google: Gemini 1.0 Pro, Gemini 1.5 Flash (May), Gemini 1.5 Flash (Sep), Gemini 1.5 Flash-8B, Gemini 1.5 Pro (May), Gemini 1.5 Pro (Sep), Gemini Experimental (Nov), Gemma 2 27B, Gemma 2 9B, and Gemma 7B, Anthropic: Claude 2.0, Claude 2.1, Claude 3 Haiku, Claude 3 Opus, Claude 3 Sonnet, Claude 3.5 Haiku, Claude 3.5 Sonnet (June), Claude 3.5 Sonnet (Oct), and Claude Instant, Mistral: Codestral, Codestral-Mamba, Ministral 3B, Ministral 8B, Mistral 7B, Mistral Large (Feb '24), Mistral Large 2 (Jul '24), Mistral Large 2 (Nov '24), Mistral Medium, Mistral NeMo, Mistral Small (Feb '24), Mistral Small (Sep '24), Mixtral 8x22B, Mixtral 8x7B, Pixtral 12B, and Pixtral Large, Cohere: Aya Expanse 32B, Aya Expanse 8B, Command, Command Light, Command-R, Command-R (Mar '24), Command-R+ (Apr '24), and Command-R+, Perplexity: PPLX-70B Online, PPLX-7B-Online, Sonar 3.1 Huge, Sonar 3.1 Large, Sonar 3.1 Small , Sonar Large, and Sonar Small, xAI: Grok Beta and Grok-1, OpenChat: OpenChat 3.5, Amazon: Nova Lite, Nova Micro, and Nova Pro, Microsoft Azure: Phi-3 Medium 14B and Phi-3 Mini, Upstage: Solar Mini, Solar Pro, and Solar Pro (Nov '24), Databricks: DBRX, NVIDIA: Llama 3.1 Nemotron 70B, IBM: Granite 3.0 2B, OpenVoice: Granite 3.0 8B, Reka AI: Reka Core, Reka Edge, Reka Flash (Feb '24), and Reka Flash, Other: LLaVA-v1.5-7B, AI21 Labs: Jamba 1.5 Large, Jamba 1.5 Mini, and Jamba Instruct, DeepSeek: DeepSeek-Coder-V2, DeepSeek-V2, and DeepSeek-V2.5, Snowflake: Arctic, Alibaba: QwQ 32B-Preview, Qwen2 72B, Qwen2.5 Coder 32B, and Qwen2.5 72B, and 01.AI: Yi-Large.