Comparison of Models: Quality, Performance & Price Analysis

Comparison and analysis of AI models across key metrics including quality, price, performance and speed (throughput tokens per second & latency), context window & others. Click on any model to see detailed metrics. For more details including relating to our methodology, see our FAQs.

Models compared: OpenAI: GPT-3.5 Turbo, GPT-3.5 Turbo (0125), GPT-3.5 Turbo (1106), GPT-3.5 Turbo Instruct, GPT-4, GPT-4 Turbo, GPT-4 Turbo (0125), and GPT-4 Vision, Google: Gemini 1.0 Pro, Gemini 1.5 Pro, and Gemma 7B, Meta: Code Llama (70B), Llama 2 Chat (13B), Llama 2 Chat (70B), Llama 2 Chat (7B), Llama 3 (70B), and Llama 3 (8B), Mistral: Mistral 7B, Mistral Large, Mistral Medium, Mistral Small, Mixtral 8x22B, and Mixtral 8x7B, Anthropic: Claude 2.0, Claude 2.1, Claude 3 Haiku, Claude 3 Opus, Claude 3 Sonnet, and Claude Instant, Cohere: Command, Command Light, Command-R, and Command-R+, Perplexity: PPLX-70B Online and PPLX-7B-Online, xAI: Grok-1, OpenChat: OpenChat 3.5, Microsoft Azure: Phi-3-mini, and Databricks: DBRX.

Model Comparison Summary

Quality: Claude 3 Opus  and  GPT-4 Vision  are the highest quality models, followed by GPT-4 Turbo, GPT-4 & Llama 3 (70B).Throughput (tokens/s): Llama 3 (8B) (216 t/s) and  Gemma 7B (165 t/s) are the fastest models, followed by Command-R, GPT-3.5 Turbo Instruct & Mixtral 8x7B.Latency (seconds): Command-R (0.15s) and   Command-R+ (0.16s) are the lowest latency models, followed by Mistral Small, Mistral Medium & Command Light.Blended Price ($/M tokens): Llama 3 (8B) ($0.14) and  Gemma 7B ($0.15) are the cheapest models, followed by OpenChat 3.5, Llama 2 Chat (7B) & Mistral 7B.Context Window Size: Gemini 1.5 Pro (1m) and  Claude 3 Opus (200k) are the largest context window models, followed by Claude 3 Sonnet, Claude 3 Haiku & Claude 2.1.

Highlights