LLM API Providers Leaderboard - Comparison of over 100 LLM endpoints

Comparison and ranking of API provider performance for over 100 AI LLM Model endpoints across performance key metrics including price, output speed, latency, context window & others. For more details including relating to our methodology, see our FAQs.

API providers compared: OpenAI, Playground AI, Microsoft Azure, Ideogram, Mistral, Amazon Bedrock, Hyperbolic, DeepSeek, Groq, FriendliAI, Together.ai, Black Forest Labs, Anthropic, Perplexity, Google, Fireworks, Cerebras, Recraft AI, Cohere, Upstage, Simplismart, Speechmatics, Fish Audio, Deepinfra, Replicate, , Genmo, Nebius, Adobe, MiniMax, CentML, Runpod, Zyphra, Rev AI, AssemblyAI, fal.ai, Reka AI, Deepgram, Gladia, Baseten, Stability.ai, Midjourney, Databricks, ElevenLabs, IBM, SambaNova, xAI, Cartesia, LMNT, Play AI, 01.AI, Alibaba Cloud, Novita, and AI21 Labs.

	Features	Model Intelligence	Price	Output tokens/s	Latency
						Further Analysis
o3-mini	200k	63	$1.93	145.1	16.33
o3-mini	200k	63	$1.93	27.2	100.12
o1	200k	62	$26.25	5.7	496.94
DeepSeek R1	64k	60	$0.96	27.3	60.46
DeepSeek R1	128k	60	$2.00	27.6	72.01
DeepSeek R1 Base	128k	60	$1.20	13.9	137.53
DeepSeek R1 Fast	128k	60	$3.00	76.3	20.14
DeepSeek R1	128k	60	$3.99	70.1	0.65
DeepSeek R1	4k	60	$0.00	10.5	161.92
DeepSeek R1	128k	60	$4.25	90.5	17.65
DeepSeek R1	64k	60	$1.16	8.0	167.04
DeepSeek R1	64k	60	$4.00	13.5	110.72
DeepSeek R1	128k	60	$7.00	94.7	20.51
o1-mini	128k	54	$1.93	186.7	10.55
o1-mini	128k	54	$5.78	190.8	15.15
DeepSeek R1 Distill Qwen 14B	64k	50	$0.15	41.6	17.43
DeepSeek R1 Distill Qwen 14B	128k	50	$1.60	145.3	6.37
Gemini 2.0 Pro Experimental (AI Studio)	2m	49	$0.00	123.5	0.56
Gemini 2.0 Flash Vertex	1m	48	$0.26	0.0	0.10
Gemini 2.0 Flash (AI Studio)	1m	48	$0.17	155.9	0.28
DeepSeek V3	66k	46	$0.48	25.9	4.54
DeepSeek V3 (FP8)	128k	46	$0.25	22.3	1.44
DeepSeek V3	128k	46	$0.75	24.1	0.92
DeepSeek V3	128k	46	$1.31	21.8	1.03
DeepSeek V3	64k	46	$0.59	6.6	1.05
DeepSeek V3	64k	46	$0.89	12.5	0.89
DeepSeek V3 (FP8)	128k	46	$1.25	10.3	1.27
DeepSeek R1 Distill Llama 70B	66k	45	$0.94	1,758.8	0.75
DeepSeek R1 Distill Llama 70B Base	128k	45	$0.38	69.3	15.21
DeepSeek R1 Distill Llama 70B	128k	45	$0.34	21.8	29.44
DeepSeek R1 Distill Llama 70B	32k	45	$0.39	17.6	21.35
DeepSeek R1 Distill Llama 70B	128k	45	$0.81	260.5	2.79
DeepSeek R1 Distill Llama 70B (Spec decoding)	128k	45	$0.81	1,695.9	1.26
DeepSeek R1 Distill Llama 70B	16k	45	$0.88	123.7	14.03
DeepSeek R1 Distill Llama 70B	128k	45	$2.00	95.8	17.97
Qwen2.5 Max	32k	45	$2.80	36.1	1.29
DeepSeek R1 Distill Qwen 32B	128k	45	$0.14	38.7	14.19
DeepSeek R1 Distill Qwen 32B	64k	45	$0.30	20.7	36.77
DeepSeek R1 Distill Qwen 32B	128k	45	$0.69	383.4	2.48
Gemini 1.5 Pro (Sep) (Vertex)	2m	45	$2.19	0.0	0.10
Gemini 1.5 Pro (Sep) (AI Studio)	2m	45	$2.19	61.8	0.76
Claude 3.5 Sonnet (Oct)	200k	44	$6.00	55.0	0.86
Claude 3.5 Sonnet (Oct) Vertex	200k	44	$6.00	88.1	1.27
Claude 3.5 Sonnet (Oct)	200k	44	$6.00	72.5	1.58
QwQ 32B-Preview	33k	43	$0.20	32.7	0.60
QwQ 32B-Preview	33k	43	$0.14	80.3	0.59
QwQ 32B-Preview	33k	43	$0.90	124.4	0.55
QwQ 32B-Preview	33k	43	$0.26	58.0	0.33
QwQ 32B-Preview	33k	43	$1.20	65.8	0.62
Gemini 2.0 Flash-Lite (Preview) (AI Studio)	1m	42	$0.13	257.8	0.26
GPT-4o (Nov '24)	128k	41	$4.38	55.1	0.58
GPT-4o (Nov '24)	128k	41	$4.38	105.0	1.16
Llama 3.3 70B	33k	41	$0.94	2,381.0	0.17
Llama 3.3 70B	128k	41	$0.40	24.9	0.53
Llama 3.3 70B	128k	41	$0.71	117.6	0.64
Llama 3.3 70B Fast	128k	41	$0.38	72.5	0.59
Llama 3.3 70B Base	128k	41	$0.20	27.8	0.96
Llama 3.3 70B	128k	41	$0.50	129.1	0.52
Llama 3.3 70B	128k	41	$0.90	115.8	0.65
Llama 3.3 70B (Turbo, FP8)	128k	41	$0.20	33.8	0.28
Llama 3.3 70B	128k	41	$0.27	24.8	0.30
Llama 3.3 70B	128k	41	$0.60	200.3	0.34
Llama 3.3 70B	128k	41	$0.39	46.3	0.92
Llama 3.3 70B (Spec decoding)	8k	41	$0.69	1,573.8	0.41
Llama 3.3 70B	128k	41	$0.64	275.0	0.36
Llama 3.3 70B	4k	41	$0.75	437.2	0.32
Llama 3.3 70B Turbo	128k	41	$0.88	184.5	0.43
GPT-4o (ChatGPT)	128k	41	$7.50	101.7	0.48
GPT-4o (Aug '24)	128k	41	$4.38	70.9	0.40
GPT-4o (Aug '24)	128k	41	$4.38	106.1	0.71
GPT-4o (May '24)	128k	41	$7.50	68.6	0.41
Llama 3.1 405B	128k	40	$9.50	18.9	0.47
Llama 3.1 405B	128k	40	$4.00	6.7	0.88
Llama 3.1 405B Standard	128k	40	$2.40	30.9	1.94
Llama 3.1 405B Latency Optimized	128k	40	$3.00	64.6	0.77
Llama 3.1 405B Base	128k	40	$1.50	33.6	0.74
Llama 3.1 405B Vertex	128k	40	$7.75	0.0	0.08
Llama 3.1 405B	128k	40	$3.00	67.1	0.88
Llama 3.1 405B	33k	40	$0.90	16.2	0.44
Llama 3.1 405B	128k	40	$7.50	37.4	0.77
Llama 3.1 405B Turbo	128k	40	$3.50	26.4	0.67
Qwen2.5 72B	131k	40	$0.40	35.7	0.56
Qwen2.5 72B	131k	40	$0.20	36.5	0.68
Qwen2.5 72B Fast	131k	40	$0.38	68.0	0.59
Qwen2.5 72B	131k	40	$0.90	89.7	0.56
Qwen2.5 72B	33k	40	$0.27	44.5	0.29
Qwen2.5 72B	8k	40	$2.50	267.3	0.38
Qwen2.5 72B Turbo	131k	40	$1.20	61.4	0.70
Qwen2.5 72B	131k	40	$0.00	18.0	1.16
Phi-4	16k	40	$0.09	37.9	0.26
Tulu3 405B	16k	40	$6.25	178.9	0.68
MiniMax-Text-01	1m	40	$0.42	42.4	0.79
Mistral Large 2 (Nov '24)	128k	38	$3.00	46.3	0.34
Mistral Large 2 (Nov '24)	128k	38	$3.00	34.1	0.50
Grok Beta	128k	38	$7.50	67.6	0.33
Pixtral Large	128k	37	$3.00	35.4	0.40
Qwen2.5 Instruct 32B Fast	128k	37	$0.00	83.0	0.58
Qwen2.5 Instruct 32B Base	128k	37	$0.00	61.0	0.58
Qwen2.5 Instruct 32B	128k	37	$0.79	392.5	0.35
Llama 3.1 Nemotron 70B Base	128k	37	$0.20	47.2	0.62
Llama 3.1 Nemotron 70B Fast	128k	37	$0.38	71.0	0.60
Llama 3.1 Nemotron 70B	128k	37	$0.27	29.7	0.32
Nova Pro	300k	37	$1.40	88.7	0.40
Mistral Large 2 (Jul '24)	128k	37	$3.00	30.6	0.37
Mistral Large 2 (Jul '24)	128k	37	$3.00	31.9	0.47
Mistral Large 2 (Jul '24)	128k	37	$3.00	34.6	0.54
Qwen2.5 Coder 32B	131k	36	$0.20	36.4	0.47
Qwen2.5 Coder 32B	33k	36	$0.90	109.2	0.54
Qwen2.5 Coder 32B	33k	36	$0.10	47.5	0.25
Qwen2.5 Coder 32B	131k	36	$0.79	385.3	0.35
Qwen2.5 Coder 32B	8k	36	$1.88	248.9	0.66
Qwen2.5 Coder 32B	131k	36	$0.80	80.8	0.55
GPT-4o mini	128k	36	$0.26	79.2	0.41
GPT-4o mini	128k	36	$0.26	178.6	0.83
DeepSeek R1 Distill Llama 8B	32k	35	$0.04	47.8	9.57
Llama 3.1 70B	128k	35	$0.40	27.5	0.58
Llama 3.1 70B Standard	128k	35	$0.72	31.3	0.70
Llama 3.1 70B Latency Optimized	128k	35	$0.90	142.4	0.36
Llama 3.1 70B Base	128k	35	$0.20	40.7	0.65
Llama 3.1 70B Fast	128k	35	$0.38	72.7	0.59
Llama 3.1 70B Vertex	128k	35	$0.00	0.0	0.09
Llama 3.1 70B	128k	35	$2.90	54.9	0.55
Llama 3.1 70B	128k	35	$0.90	139.8	0.57
Llama 3.1 70B (Turbo, FP8)	128k	35	$0.20	44.7	0.25
Llama 3.1 70B	128k	35	$0.27	36.3	0.28
Llama 3.1 70B	128k	35	$0.60	217.8	0.35
Llama 3.1 70B	32k	35	$0.35	81.2	0.98
Llama 3.1 70B	64k	35	$0.75	131.9	0.50
Llama 3.1 70B	128k	35	$1.50	59.3	0.45
Llama 3.1 70B Turbo	128k	35	$0.88	200.7	0.43
Llama 3.1 70B	128k	35	$0.90	126.5	0.57
Mistral Small 3	32k	35	$0.15	145.8	0.31
Mistral Small 3	32k	35	$0.90	92.6	0.57
Mistral Small 3	32k	35	$0.09	85.4	0.21
Mistral Small 3	32k	35	$0.80	99.5	0.23
Claude 3 Opus	200k	35	$30.00	26.5	1.32
Claude 3 Opus Vertex	200k	35	$30.00	27.2	3.76
Claude 3 Opus	200k	35	$30.00	27.8	0.98
Claude 3.5 Haiku Standard	200k	35	$1.60	47.9	0.87
Claude 3.5 Haiku Latency Optimized	200k	35	$2.00	98.3	0.55
Claude 3.5 Haiku Vertex	200k	35	$1.60	67.4	0.70
Claude 3.5 Haiku	200k	35	$1.60	65.6	0.57
Gemini 1.5 Pro (May) (Vertex)	2m	34	$2.19	0.0	0.10
Gemini 1.5 Pro (May) (AI Studio)	2m	34	$2.19	70.0	0.75
Qwen Turbo	1m	34	$0.09	85.7	1.12
Llama 3.2 90B (Vision)	128k	33	$0.72	53.0	0.39
Llama 3.2 90B (Vision) Vertex	128k	33	$0.00	0.0	0.08
Llama 3.2 90B (Vision)	128k	33	$0.90	77.6	0.52
Llama 3.2 90B (Vision)	33k	33	$0.36	36.9	0.27
Llama 3.2 90B (Vision)	8k	33	$0.90	255.8	0.31
Llama 3.2 90B (Vision) Turbo	128k	33	$1.20	54.8	0.30
Mistral Saba	32k	32	$0.30	94.2	0.31
Jamba 1.5 Large	256k	29	$3.50	0.0	0.00
Jamba 1.5 Large	256k	29	$3.50	50.9	0.87
Gemini 1.5 Flash (May) (Vertex)	1m	28	$0.13	0.0	0.11
Gemini 1.5 Flash (May) (AI Studio)	1m	28	$0.13	293.6	0.30
Nova Micro	130k	28	$0.06	194.6	0.35
Yi-Large	32k	28	$3.00	68.7	0.43
Codestral (Jan '25)	256k	28	$0.45	211.3	0.27
Codestral (Jan '25) Vertex	128k	28	$0.45	152.0	0.14
Llama 3 70B	8k	27	$1.18	44.6	0.40
Llama 3 70B	8k	27	$0.40	32.2	0.63
Llama 3 70B	8k	27	$2.86	53.3	0.42
Llama 3 70B	8k	27	$2.90	19.0	0.83
Llama 3 70B	8k	27	$0.90	159.9	0.50
Llama 3 70B	8k	27	$0.27	43.6	0.28
Llama 3 70B	8k	27	$0.57	22.5	5.64
Llama 3 70B	8k	27	$0.64	336.8	0.28
Llama 3 70B (Reference, FP16)	8k	27	$0.90	200.9	0.41
Llama 3 70B (Turbo, FP8)	8k	27	$0.88	16.5	0.31
Mistral Small (Sep '24)	33k	27	$0.30	84.3	0.32
Mistral Large (Feb '24)	33k	26	$6.00	36.4	0.40
Mistral Large (Feb '24)	33k	26	$6.00	43.8	0.38
Mistral Large (Feb '24)	33k	26	$6.00	38.3	0.50
Mixtral 8x22B	65k	26	$3.00	75.4	0.28
Mixtral 8x22B Base	65k	26	$0.60	87.5	0.62
Mixtral 8x22B Fast	65k	26	$1.05	103.4	0.63
Mixtral 8x22B	65k	26	$1.20	94.9	0.51
Mixtral 8x22B	65k	26	$1.20	72.7	0.82
Phi-3 Medium 14B	128k	25	$0.30	52.4	0.43
Mistral Medium	33k	24	$4.09	43.7	0.36
Llama 3.1 8B	33k	24	$0.10	2,208.3	0.27
Llama 3.1 8B	128k	24	$0.10	121.2	0.45
Llama 3.1 8B	128k	24	$0.22	88.9	0.38
Llama 3.1 8B Fast	128k	24	$0.04	185.3	0.52
Llama 3.1 8B Base	128k	24	$0.03	66.0	0.57
Llama 3.1 8B Vertex	128k	24	$0.00	0.0	0.09
Llama 3.1 8B	128k	24	$0.20	203.2	0.30
Llama 3.1 8B	128k	24	$0.04	50.0	0.28
Llama 3.1 8B	128k	24	$0.10	482.4	0.30
Llama 3.1 8B	16k	24	$0.05	67.0	0.63
Llama 3.1 8B	128k	24	$0.06	750.5	0.33
Llama 3.1 8B	16k	24	$0.13	633.0	0.27
Llama 3.1 8B Turbo	128k	24	$0.18	218.9	0.28
Llama 3.1 8B	128k	24	$0.15	471.1	0.51
Pixtral 12B	128k	23	$0.15	102.5	0.29
Pixtral 12B	128k	23	$0.10	67.7	0.50
Mistral Small (Feb '24)	33k	23	$1.50	149.5	0.27
Mistral Small (Feb '24)	33k	23	$1.50	53.2	0.40
Ministral 8B	128k	22	$0.10	140.7	0.33
Llama 3.2 11B (Vision)	128k	22	$0.16	145.3	0.35
Llama 3.2 11B (Vision)	128k	22	$0.20	148.8	0.48
Llama 3.2 11B (Vision)	128k	22	$0.06	45.0	0.23
Llama 3.2 11B (Vision)	8k	22	$0.18	749.7	0.31
Llama 3.2 11B (Vision) Turbo	128k	22	$0.18	145.1	0.22
Command-R+	128k	21	$6.00	47.0	0.49
Command-R+	128k	21	$4.38	73.7	0.23
Codestral (May '24)	33k	20	$0.30	84.4	0.28
Aya Expanse 32B	128k	20	$0.75	120.9	0.16
DBRX	33k	20	$1.13	66.5	0.55
DBRX	33k	20	$1.20	83.4	0.33
Ministral 3B	128k	20	$0.04	220.1	0.30
Mistral NeMo	128k	20	$0.15	118.2	0.26
Mistral NeMo Fast	128k	20	$0.12	160.7	0.54
Mistral NeMo Base	128k	20	$0.06	25.5	0.79
Mistral NeMo	128k	20	$0.06	68.1	0.25
DeepSeek R1 Distill Qwen 1.5B	128k	20	$0.18	368.9	6.39
OpenChat 3.5	8k	16	$0.06	80.9	0.30
Command-R (Mar '24)	128k	15	$0.75	108.6	0.35
Command-R (Mar '24)	128k	15	$0.75	170.8	0.17
Command-R (Mar '24)	128k	15	$0.75	79.6	0.46
Codestral-Mamba	256k	14	$0.25	94.8	0.45
o1-preview	128k		$26.25	113.7	32.03
o1-preview	128k		$28.88	105.8	34.37
Llama 3.2 3B	128k		$0.10	202.4	0.47
Llama 3.2 3B	128k		$0.15	72.2	0.34
Llama 3.2 3B Base	128k		$0.01	122.8	0.52
Llama 3.2 3B	128k		$0.10	332.8	0.47
Llama 3.2 3B	128k		$0.02	146.4	0.21
Llama 3.2 3B	32k		$0.04	120.2	0.62
Llama 3.2 3B	8k		$0.06	1,608.2	0.33
Llama 3.2 3B	4k		$0.10	1,434.2	0.23
Llama 3.2 3B Turbo	128k		$0.06	53.6	0.41
Llama 3.2 1B	128k		$0.10	118.0	0.32
Llama 3.2 1B Base	128k		$0.01	264.7	0.50
Llama 3.2 1B	128k		$0.01	155.4	0.25
Llama 3.2 1B	8k		$0.04	3,021.1	0.51
Llama 3.2 1B	4k		$0.05	1,568.0	0.30
Gemini 2.0 Flash (exp) (AI Studio)	1m		$0.00	156.9	0.26
Gemini 1.5 Flash (Sep) (Vertex)	1m		$0.13	0.0	0.10
Gemini 1.5 Flash (Sep) (AI Studio)	1m		$0.13	192.1	0.40
Gemma 2 27B	8k		$0.80	82.0	0.33
Gemma 2 9B Base	8k		$0.03	165.8	0.52
Gemma 2 9B	8k		$0.04	45.8	0.34
Gemma 2 9B	8k		$0.20	650.2	0.23
Gemma 2 9B	8k		$0.30	135.3	0.20
Claude 3.5 Sonnet (June)	200k		$6.00	46.4	1.50
Claude 3.5 Sonnet (June) Vertex	200k		$6.00	75.0	1.07
Claude 3.5 Sonnet (June)	200k		$6.00	85.5	1.02
Claude 3 Haiku	200k		$0.50	91.5	0.84
Claude 3 Haiku	200k		$0.50	136.5	0.54
Mixtral 8x7B	33k		$0.70	102.5	0.30
Mixtral 8x7B	33k		$0.51	47.5	0.35
Mixtral 8x7B Fast	33k		$0.23	163.5	0.51
Mixtral 8x7B Base	33k		$0.12	133.1	0.53
Mixtral 8x7B	33k		$0.50	173.3	0.47
Mixtral 8x7B	33k		$0.24	103.5	0.20
Mixtral 8x7B	33k		$0.24	568.7	0.27
Mixtral 8x7B	33k		$0.63	93.9	0.42
Mixtral 8x7B	33k		$0.60	105.7	0.36
Nova Lite	300k		$0.10	140.9	0.37
Command-R+ (Apr '24)	128k		$6.00	46.7	0.50
Command-R+ (Apr '24)	128k		$6.00	76.2	0.26
Command-R+ (Apr '24)	128k		$6.00	50.3	0.60
Aya Expanse 8B	8k		$0.75	166.7	0.14
Command-R	128k		$0.75	108.3	0.36
Command-R	128k		$0.26	94.0	0.17
Jamba 1.5 Mini	256k		$0.25	0.0	0.00
Jamba 1.5 Mini	256k		$0.25	82.6	0.52
Qwen2 72B	33k		$0.90	67.5	0.37
GPT-4 Turbo	128k		$15.00	32.6	1.11
GPT-4 Turbo	128k		$15.00	49.2	4.07
Llama 3 8B	8k		$0.10	78.0	0.40
Llama 3 8B	8k		$0.38	103.5	0.32
Llama 3 8B	8k		$0.20	230.4	0.46
Llama 3 8B	8k		$0.04	101.6	0.19
Llama 3 8B	8k		$0.04	45.7	0.75
Llama 3 8B	8k		$0.06	1,200.0	0.32
Llama 3 8B	8k		$0.20	278.0	0.26
Llama 2 Chat 7B	4k		$0.10	123.6	0.42
Gemini 1.0 Pro (AI Studio)	33k		$0.75	103.6	1.24
Claude 3 Sonnet	200k		$6.00	45.7	0.74
Claude 3 Sonnet	200k		$6.00	58.1	0.58
Claude 2.1	200k		$12.00	29.5	1.71
Claude 2.1	200k		$12.00	13.7	0.82
Claude 2.0	100k		$12.00	29.5	0.81
Mistral 7B	8k		$0.25	128.7	0.26
Mistral 7B	8k		$0.16	91.7	0.33
Mistral 7B	8k		$0.04	93.2	0.27
Mistral 7B	32k		$0.06	99.8	0.95
Mistral 7B	8k		$0.20	176.5	0.18
Jamba Instruct	256k		$0.55	0.0	0.00
Jamba Instruct	256k		$0.55	75.2	1.08

Key definitions

Back to Navigation

Context window: Maximum number of combined input & output tokens. Output tokens commonly have a significantly lower limit (varied by model).

Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).

Latency: Time to first token of tokens received, in seconds, after API request sent. For models which do not support streaming, this represents time to receive the completion.

Price: Price per token, represented as USD per million Tokens. Price is a blend of Input & Output token prices (3:1 ratio).

Output Price: Price per token generated by the model (received from the API), represented as USD per million Tokens.

Input Price: Price per token included in the request/message sent to the API, represented as USD per million Tokens.

Time period: Metrics are 'live' and are based on the past 72 hours of measurements, measurements are taken 8 times a day for single requests and 2 times per day for parallel requests.