Grok Beta: Intelligence, Performance & Price Analysis

Analysis of xAI's Grok Beta and comparison to other AI models across key metrics including quality, price, performance (tokens per second & time to first token), context window & more. Click on any model to compare API providers for that model. For more details including relating to our methodology, see our FAQs.

This is Grok-2 Beta, weights and any other information are not openly released until the next version of Grok is out! https://x.com/elonmusk/status/1842248588149117013

For analysis of API providers see

Comparison Summary

Intelligence:

Grok Beta is of higher quality compared to average, with a MMLU score of 0.703 and a Intelligence Index across evaluations of 38.

Price:

Grok Beta is more expensive compared to average with a price of $7.50 per 1M Tokens (blended 3:1).
Grok Beta Input token price: $5.00, Output token price: $15.00 per 1M Tokens.

Speed:

Grok Beta is slower compared to average, with a output speed of 66.8 tokens per second.

Latency:

Grok Beta has a lower latency compared to average, taking 0.31s to receive the first token (TTFT).

Context Window:

Grok Beta has a smaller context windows than average, with a context window of 130k tokens.

Highlights

Intelligence

Artificial Analysis Intelligence Index; Higher is better

Speed

Output Tokens per Second; Higher is better

Price

USD per 1M Tokens; Lower is better

Grok Beta Model Details

Creator
Context Window	128k tokens (~192 A4 pages of size 12 Arial font)
Release Date	August, 2024
Image Input Support	Yes
Open Source (Weights)	No

Further details

Model Name	Creator	License	Context Window

o1	OpenAI	Proprietary	200k
o3-mini	OpenAI	Proprietary	200k
o1-preview	OpenAI	Proprietary	128k
o1-mini	OpenAI	Proprietary	128k
GPT-4o (Aug '24)	OpenAI	Proprietary	128k
GPT-4o (May '24)	OpenAI	Proprietary	128k
GPT-4o (Nov '24)	OpenAI	Proprietary	128k
GPT-4o mini	OpenAI	Proprietary	128k
o3-mini (high)	OpenAI	Proprietary	200k
o3	OpenAI	Proprietary	128k
GPT-4o (ChatGPT)	OpenAI	Proprietary	128k
GPT-4.5 (Preview)	OpenAI	Proprietary	128k
GPT-4 Turbo	OpenAI	Proprietary	128k
GPT-4	OpenAI	Proprietary	8k

Llama 3.3 Instruct 70B	Meta	Open	128k
Llama 3.1 Instruct 405B	Meta	Open	128k
Llama 3.2 Instruct 90B (Vision)	Meta	Open	128k
Llama 3.1 Instruct 70B	Meta	Open	128k
Llama 3.2 Instruct 11B (Vision)	Meta	Open	128k
Llama 3.1 Instruct 8B	Meta	Open	128k
Llama 3.2 Instruct 3B	Meta	Open	128k
Llama 3.2 Instruct 1B	Meta	Open	128k
Llama 3 Instruct 70B	Meta	Open	8k
Llama 3 Instruct 8B	Meta	Open	8k
Llama 2 Chat 7B	Meta	Open	4k

Gemini 2.0 Pro Experimental (Feb '25)	Google	Proprietary	2m
Gemini 2.0 Flash (Feb '25)	Google	Proprietary	1m
Gemini 2.0 Flash (experimental)	Google	Proprietary	1m
Gemini 1.5 Pro (Sep '24)	Google	Proprietary	2m
Gemini 1.5 Flash (Sep '24)	Google	Proprietary	1m
Gemini 1.5 Pro (May '24)	Google	Proprietary	2m
Gemma 2 27B	Google	Open	8k
Gemma 2 9B	Google	Open	8k
Gemini 1.5 Flash-8B	Google	Proprietary	1m
Gemini 2.0 Flash-Lite (Feb '25)	Google	Proprietary	1m
Gemma 3 27B Instruct	Google	Open	128k
Gemma 3 1B Instruct	Google	Open	32k
Gemini 2.0 Flash-Lite (Preview)	Google	Proprietary	1m
Gemini 1.5 Flash (May '24)	Google	Proprietary	1m
Gemini 1.0 Pro	Google	Proprietary	33k

Claude 3.5 Sonnet (Oct '24)	Anthropic	Proprietary	200k
Claude 3.5 Sonnet (June '24)	Anthropic	Proprietary	200k
Claude 3 Opus	Anthropic	Proprietary	200k
Claude 3.5 Haiku	Anthropic	Proprietary	200k
Claude 3 Haiku	Anthropic	Proprietary	200k
Claude 3.7 Sonnet (Extended Thinking)	Anthropic	Proprietary	200k
Claude 3.7 Sonnet (Standard)	Anthropic	Proprietary	200k
Claude 3 Sonnet	Anthropic	Proprietary	200k
Claude 2.0	Anthropic	Proprietary	100k
Claude 2.1	Anthropic	Proprietary	200k

Pixtral Large	Mistral	Open	128k
Mistral Large 2 (Nov '24)	Mistral	Open	128k
Mistral Large 2 (Jul '24)	Mistral	Open	128k
Mistral Small 3	Mistral	Open	32k
Mistral Small (Sep '24)	Mistral	Open	33k
Mixtral 8x22B Instruct	Mistral	Open	65k
Pixtral 12B (2409)	Mistral	Open	128k
Ministral 8B	Mistral	Open	128k
Mistral NeMo	Mistral	Open	128k
Ministral 3B	Mistral	Proprietary	128k
Mixtral 8x7B Instruct	Mistral	Open	33k
Codestral-Mamba	Mistral	Open	256k
Codestral (Jan '25)	Mistral	Proprietary	256k
Mistral Small 3.1	Mistral	Open	128k
Mistral Saba	Mistral	Proprietary	32k
Mistral Small (Feb '24)	Mistral	Proprietary	33k
Mistral Large (Feb '24)	Mistral	Proprietary	33k
Mistral 7B Instruct	Mistral	Open	8k
Codestral (May '24)	Mistral	Open	33k
Mistral Medium	Mistral	Proprietary	33k

DeepSeek R1	DeepSeek	Open	128k
DeepSeek R1 Distill Llama 70B	DeepSeek	Open	128k
DeepSeek V3	DeepSeek	Open	128k
DeepSeek-V2.5 (Dec '24)	DeepSeek	Open	128k
DeepSeek-Coder-V2	DeepSeek	Open	128k
DeepSeek LLM 67B Chat (V1)	DeepSeek	Open	4k
DeepSeek R1 Distill Qwen 14B	DeepSeek	Open	128k
DeepSeek Coder V2 Lite Instruct	DeepSeek	Open	128k
DeepSeek-V2.5	DeepSeek	Open	128k
DeepSeek R1 Distill Llama 8B	DeepSeek	Open	128k
DeepSeek R1 Distill Qwen 32B	DeepSeek	Open	128k
DeepSeek-V2-Chat	DeepSeek	Open	128k
DeepSeek R1 Distill Qwen 1.5B	DeepSeek	Open	128k

Sonar Pro	Perplexity	Proprietary	200k
Sonar	Perplexity	Proprietary	127k

Grok Beta	xAI	Proprietary	128k
Grok 3 Reasoning Beta	xAI	Proprietary	1m
Grok 3 mini Reasoning	xAI	Proprietary	1m
Grok 3	xAI	Proprietary	1m
Grok 3 mini	xAI	Proprietary	1m

Nova Pro	Amazon	Proprietary	300k
Nova Lite	Amazon	Proprietary	300k
Nova Micro	Amazon	Proprietary	130k

Phi-4	Microsoft Azure	Open	16k
Phi-4 Multimodal Instruct	Microsoft Azure	Open	128k
Phi-3 Medium Instruct 14B	Microsoft Azure	Open	128k
Phi-4 Mini Instruct	Microsoft Azure	Open	128k

LFM 40B	Liquid AI	Proprietary	32k

Solar Mini	Upstage	Open	4k

DBRX Instruct	Databricks	Open	33k

MiniMax-Text-01	MiniMax	Open	4m

Llama 3.1 Nemotron Instruct 70B	NVIDIA	Open	128k

Llama 3.1 Tulu3 405B	Allen Institute for AI	Open	128k

Reka Flash (Sep '24)	Reka AI	Proprietary	128k
Reka Core	Reka AI	Proprietary	128k
Reka Flash (Feb '24)	Reka AI	Proprietary	128k
Reka Edge	Reka AI	Proprietary	128k
Reka Flash 3	Reka AI	Open	128k

Command-R+ (Aug '24)	Cohere	Open	128k
Command-R+ (Apr '24)	Cohere	Open	128k
Command-R (Mar '24)	Cohere	Open	128k
Aya Expanse 32B	Cohere	Open	128k
Aya Expanse 8B	Cohere	Open	8k
Command A	Cohere	Open	256k
Command-R (Aug '24)	Cohere	Open	128k

Jamba 1.5 Large	AI21 Labs	Open	256k
Jamba 1.6 Mini	AI21 Labs	Open	256k
Jamba 1.5 Mini	AI21 Labs	Open	256k
Jamba 1.6 Large	AI21 Labs	Open	256k
Jamba Instruct	AI21 Labs	Proprietary	256k

Qwen2.5 Max	Alibaba	Proprietary	32k
Qwen2.5 Instruct 72B	Alibaba	Open	131k
Qwen2.5 Coder Instruct 32B	Alibaba	Open	131k
Qwen Turbo	Alibaba	Proprietary	1m
Qwen2 Instruct 72B	Alibaba	Open	131k
QwQ 32B	Alibaba	Open	131k
Qwen2.5 Instruct 32B	Alibaba	Open	128k
Qwen2.5 Coder Instruct 7B	Alibaba	Open	131k
Qwen Chat 72B	Alibaba	Open	34k
Qwen1.5 Chat 110B	Alibaba	Open	32k
QwQ 32B-Preview	Alibaba	Open	33k

Yi-Large	01.AI	Proprietary	32k

OpenChat 3.5 (1210)	OpenChat	Open	8k