Comparison of Models: Quality, Performance & Price Analysis
Comparison and analysis of AI models across key performance metrics including quality, price, output speed, latency, context window & others. Click on any model to see detailed metrics. For more details including relating to our methodology, see our FAQs.
Model Comparison Summary
o1-preview and o1-mini are the highest quality models, followed by GPT-4o (Aug 6) & 
Claude 3.5 Sonnet.Output Speed (tokens/s):
Gemma 7B (1024 t/s) and Gemini 1.5 Flash (216 t/s) are the fastest models, followed by 
Llama 3.1 8B & 
Jamba 1.5 Mini.Latency (seconds):
Sonar 3.1 Small (0.17s) and Sonar Small (0.18s) are the lowest latency models, followed by Sonar 3.1 Large & Sonar Large.Price ($ per M tokens):
OpenChat 3.5 ($0.06) and Gemma 7B ($0.07) are the cheapest models, followed by Gemini 1.5 Flash & Llama 3.1 8B.Context Window: Gemini 1.5 Pro (2m) and Gemini 1.5 Flash (1m) are the largest context window models, followed by 
Codestral-Mamba & Jamba 1.5 Large.Highlights
Command-R+ (08-2024)
Phi-3 Medium Instruct 14B
DBRX Instruct
Reka Core
DeepSeek-Coder-V2
Qwen2 Instruct 72B
Yi-Large
Models compared: OpenAI: GPT-3.5 Turbo, GPT-3.5 Turbo (0125), GPT-3.5 Turbo (1106), GPT-3.5 Turbo Instruct, GPT-4, GPT-4 Turbo, GPT-4 Turbo (0125), GPT-4 Vision, GPT-4o, GPT-4o (Aug 6), GPT-4o mini, o1-mini, and o1-preview, Meta: Code Llama 70B, Llama 2 Chat 13B, Llama 2 Chat 70B, Llama 2 Chat 7B, Llama 3 70B, Llama 3 8B, Llama 3.1 405B, Llama 3.1 70B, and Llama 3.1 8B, Google: Gemini 1.0 Pro, Gemini 1.5 Flash, Gemini 1.5 Pro, Gemma 2 27B, Gemma 2 9B, and Gemma 7B, Anthropic: Claude 2.0, Claude 2.1, Claude 3 Haiku, Claude 3 Opus, Claude 3 Sonnet, Claude 3.5 Sonnet, and Claude Instant, Mistral: Codestral, Codestral-Mamba, Mistral 7B, Mistral Large, Mistral Large 2, Mistral Medium, Mistral NeMo, Mistral Small, Mixtral 8x22B, Mixtral 8x7B, and Pixtral 12B, Cohere: Command, Command Light, Command-R (03-2024), Command-R (08-2024), Command-R+ (04-2024), and Command-R+ (08-2024), Perplexity: PPLX-70B Online, PPLX-7B-Online, Sonar 3.1 Large, Sonar 3.1 Small , Sonar Large, and Sonar Small, xAI: Grok-1, OpenChat: OpenChat 3.5, Microsoft Azure: Phi-3 Medium 14B and Phi-3 Mini, Databricks: DBRX, Reka AI: Reka Core, Reka Edge, and Reka Flash, Other: LLaVA-v1.5-7B, Glaive: Reflection Llama 3.1 - 70B and Reflection Llama 3.1 70B v2, AI21 Labs: Jamba 1.5 Large, Jamba 1.5 Mini, and Jamba Instruct, DeepSeek: DeepSeek-Coder-V2, DeepSeek-V2, and DeepSeek-V2.5, Snowflake: Arctic, Alibaba: Qwen2 72B, and 01.AI: Yi-Large.