LLM API Providers Leaderboard - Comparison of over 100 LLM endpoints

Comparison and ranking of API provider performance for over 100 AI LLM Model endpoints across performance key metrics including price, output speed, latency, context window & others. For more details including relating to our methodology, see our FAQs.

API providers compared: OpenAI, Mistral, Microsoft Azure, Amazon Bedrock, Groq, Together.ai, Anthropic, Perplexity, Google, Fireworks, Baseten, Cohere, Lepton AI, Speechmatics, Deepinfra, Replicate, Runpod, Rev AI, fal.ai, AssemblyAI, DeepSeek, Reka AI, Deepgram, Gladia, Stability.ai, Midjourney, Databricks, ElevenLabs, IBM, OctoAI, Cartesia, LMNT, 01.AI, and AI21 Labs.

	Context	Model Quality	Price	Output tokens/s	Latency
						Further Analysis
GPT-4o	128k	100	$7.50	82.3	0.45
GPT-4 Turbo	128k	94	$15.00	28.0	0.69
GPT-4 Turbo	128k	94	$15.00	31.0	0.57
GPT-4o mini	128k	88	$0.26	98.9	0.55
GPT-4	8k	84	$37.50	24.8	0.84
GPT-4	8k	84	$37.50	26.6	0.55
GPT-3.5 Turbo Instruct	4k	60	$1.63	74.6	0.33
GPT-3.5 Turbo Instruct	4k	60	$1.63	143.8	0.59
GPT-3.5 Turbo	16k	59	$0.75	92.5	0.44
GPT-3.5 Turbo	16k	59	$0.75	71.8	0.34
Gemini 1.5 Pro	2m	95	$5.25	57.7	1.07
Gemini 1.5 Flash	1m	84	$0.53	166.2	1.02
Gemma 2 27B	8k	78	$0.80	76.9	0.49
Gemma 2 9B	8k	71	$0.20	116.7	0.30
Gemma 2 9B	8k	71	$0.09	53.9	0.22
Gemma 2 9B	8k	71	$0.20	576.6	0.23
Gemma 2 9B	8k	71	$0.30	100.0	0.65
Gemini 1.0 Pro	33k	62	$0.75	87.0	2.10
Gemma 7B	8k	45	$0.20	231.1	0.27
Gemma 7B	8k	45	$0.07	64.3	0.32
Gemma 7B	8k	45	$0.07	1,039.1	0.85
Gemma 7B	8k	45	$0.20	141.0	0.28
Llama 3.1 405B	128k	100	$9.50	28.0	1.70
Llama 3.1 405B	128k	100	$4.50	25.2	0.43
Llama 3.1 405B	128k	100	$2.80	32.3	1.03
Llama 3.1 405B	128k	100	$8.00
Llama 3.1 405B	128k	100	$3.00	28.8	0.57
Llama 3.1 405B	33k	100	$8.75	10.6	0.49
Llama 3.1 405B	128k	100	$15.00	22.9	0.63
Llama 3.1 405B	4k	100	$5.00	68.8	0.75
Llama 3.1 70B	128k	95	$0.90	50.9	0.28
Llama 3.1 70B	128k	95	$0.80	32.6	0.89
Llama 3.1 70B	128k	95	$0.90	84.0	0.33
Llama 3.1 70B	128k	95	$0.58	30.3	0.40
Llama 3.1 70B	8k	95	$0.64	249.0	0.56
Llama 3.1 70B	128k	95	$1.50	55.7	0.41
Llama 3.1 70B	33k	95	$0.88	73.4	1.06
Llama 3 70B	8k	83	$1.18	76.9	1.75
Llama 3 70B	8k	83	$2.86
Llama 3 70B	8k	83	$0.90	66.4	0.28
Llama 3 70B	8k	83	$0.80	12.3	0.92
Llama 3 70B	8k	83	$5.67	18.2	3.23
Llama 3 70B	8k	83	$0.90	92.2	0.34
Llama 3 70B	8k	83	$0.58	20.4	0.40
Llama 3 70B	8k	83	$0.64	320.0	0.29
Llama 3 70B	8k	83	$1.50	59.0	0.48
Llama 3 70B	8k	83	$1.00	39.9	0.32
Llama 3 70B	8k	83	$0.88	127.6	0.66
Llama 3 70B	8k	83	$0.88	87.1	0.70
Llama 3.1 8B	128k	66	$0.38	92.8	0.38
Llama 3.1 8B	128k	66	$0.15	93.6	0.22
Llama 3.1 8B	128k	66	$0.70	131.6	0.47
Llama 3.1 8B	128k	66	$0.20	304.1	0.22
Llama 3.1 8B	128k	66	$0.09	90.1	0.21
Llama 3.1 8B	8k	66	$0.06	738.8	0.42
Llama 3.1 8B	33k	66	$0.18	151.5	0.36
Llama 3 8B	8k	64	$0.10	77.4	1.56
Llama 3 8B	8k	64	$0.38	74.2	0.31
Llama 3 8B	8k	64	$0.38	94.2	0.36
Llama 3 8B	8k	64	$0.15	171.7	0.20
Llama 3 8B	8k	64	$0.07	93.1	0.76
Llama 3 8B	8k	64	$0.55	77.1	0.98
Llama 3 8B	8k	64	$0.20	218.5	0.26
Llama 3 8B	8k	64	$0.06	88.1	0.22
Llama 3 8B	8k	64	$0.06	1,197.1	0.37
Llama 3 8B	8k	64	$0.20	148.0	0.24
Llama 3 8B	8k	64	$0.20	246.7	0.43
Llama 2 Chat 70B	4k	57	$1.18	53.6	1.54
Llama 2 Chat 70B	4k	57	$2.10	44.8	0.51
Llama 2 Chat 70B	4k	57	$0.90	171.4	0.21
Llama 2 Chat 70B	4k	57	$1.60	17.9	3.18
Llama 2 Chat 70B	4k	57	$0.90	67.4	0.38
Llama 2 Chat 70B	4k	57	$0.90	34.2	0.85
Mistral Large 2	128k	91	$4.50	30.4	0.44
Llama 2 Chat 13B	4k	39	$0.20	84.7	1.48
Llama 2 Chat 13B	4k	39	$0.81
Llama 2 Chat 13B	4k	39	$0.20	171.7	0.20
Llama 2 Chat 13B	4k	39	$0.84	44.8	1.53
Llama 2 Chat 13B	4k	39	$0.20	102.1	0.32
Llama 2 Chat 13B	4k	39	$0.30	46.6	0.45
Llama 2 Chat 7B	4k	29	$0.10	148.3	1.24
Llama 2 Chat 7B	4k	29	$0.56	74.4	1.04
Llama 2 Chat 7B	4k	29	$0.20
Llama 2 Chat 7B	4k	29	$0.20	91.5	0.44
Codestral	33k		$1.50	53.2	0.31
Codestral-Mamba	256k		$0.25	95.6	0.43
Mistral Large	33k	76	$6.00	40.3	0.49
Mistral Large	33k	76	$6.00	35.1	0.43
Mistral Large	33k	76	$6.00	24.5	2.61
Mixtral 8x22B	65k	71	$3.00	67.6	0.37
Mixtral 8x22B	65k	71	$1.20	90.1	0.27
Mixtral 8x22B	65k	71	$1.20	78.6	0.28
Mixtral 8x22B	65k	71	$0.65	40.7	0.27
Mixtral 8x22B	65k	71	$1.20	44.1	0.49
Mistral Small	33k	71	$1.50	54.4	0.34
Mistral Small	33k	71	$1.50	88.3	1.04
Mistral Medium	33k	70	$4.05	37.9	0.66
Mistral NeMo	128k	64	$0.30	188.0	0.32
Mixtral 8x7B	33k	61	$0.70	86.0	0.39
Mixtral 8x7B	33k	61	$0.47	99.7	1.27
Mixtral 8x7B	33k	61	$0.51	45.3	0.39
Mixtral 8x7B	33k	61	$0.45	79.3	0.26
Mixtral 8x7B	33k	61	$0.50	71.5	0.43
Mixtral 8x7B	33k	61	$0.50	247.8	0.25
Mixtral 8x7B	33k	61	$0.24	57.1	0.24
Mixtral 8x7B	33k	61	$0.24	548.0	0.22
Mixtral 8x7B	33k	61	$0.63	89.8	0.48
Mixtral 8x7B	16k	61	$0.60	119.3	0.25
Mixtral 8x7B	33k	61	$0.60	91.8	0.38
Mistral 7B	33k	40	$0.25	100.0	0.38
Mistral 7B	33k	40	$0.10	81.3	1.35
Mistral 7B	33k	40	$0.16	71.6	0.34
Mistral 7B	33k	40	$0.15	155.2	0.22
Mistral 7B	33k	40	$0.07	96.6	0.90
Mistral 7B	33k	40	$0.20	173.3	0.25
Mistral 7B	33k	40	$0.06	114.7	0.20
Mistral 7B	16k	40	$0.20	163.2	0.26
Mistral 7B	8k	40	$0.20	72.3	0.36
Mistral 7B	4k	40	$0.20
Claude 3.5 Sonnet	200k	98	$6.00	78.7	1.14
Claude 3 Opus	200k	93	$30.00	24.7	1.79
Claude 3 Opus	200k	93	$30.00	26.8	2.36
Claude 3 Sonnet	200k	80	$6.00	64.3	0.81
Claude 3 Sonnet	200k	80	$6.00	62.8	1.00
Claude 3 Haiku	200k	74	$0.50	120.7	0.43
Claude 3 Haiku	200k	74	$0.50	151.4	0.63
Claude 2.0	100k	70	$12.00	39.9	1.13
Claude Instant	100k	63	$1.20	77.5	0.58
Claude Instant	100k	63	$1.20	97.4	0.59
Claude 2.1	200k	55	$12.00	37.4	1.69
Claude 2.1	200k	55	$12.00	40.8	1.12
Command Light	4k		$0.38	33.3	0.58
Command Light	4k		$0.38	67.1	0.23
Command	4k		$1.63	23.4	0.49
Command	4k		$1.25	24.6	0.37
Command-R+	128k	75	$6.00	52.0	0.37
Command-R+	128k	75	$6.00	66.2	0.49
Command-R	128k	63	$0.75	147.6	0.20
Command-R	128k	63	$0.75	101.9	0.46
Sonar Large	33k		$1.00	54.8	0.29
Sonar Small	33k		$0.20	157.2	0.23
OpenChat 3.5	8k	50	$0.07	65.3	0.34
OpenChat 3.5	8k	50	$0.20	73.6	0.38
Phi-3 Medium 14B	4k		$0.14	74.0	0.21
DBRX	33k	62	$1.20	65.7	0.31
DBRX	33k	62	$1.13	87.4	0.48
DBRX	33k	62	$1.20	95.6	0.40
Reka Core	128k	90	$6.00	15.8	1.34
Reka Flash	128k	78	$1.10	31.2	0.84
Reka Edge	64k	60	$0.55	48.9	0.84
Jamba Instruct	256k	63	$0.55	66.8	0.45
Jamba Instruct	256k	63	$0.55	66.9	1.12
DeepSeek-Coder-V2	128k		$0.17	16.5	1.24
DeepSeek-V2	128k	82	$0.17	16.8	1.15
Arctic	4k	55	$2.40	72.0	0.62
Qwen2 72B	33k	83	$0.90	78.4	0.35
Qwen2 72B	33k	83	$0.61	39.8	0.30
Qwen2 72B	33k	83	$0.90	50.9	0.46
Yi-Large	32k	81	$3.00	69.6	2.11
Yi-Large	32k	81	$3.00	74.2	0.35

Key definitions

Back to Navigation

Quality: Index represents normalized average relative performance across Chatbot arena, MMLU & MT-Bench.

Context window: Maximum number of combined input & output tokens. Output tokens commonly have a significantly lower limit (varied by model).

Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API).

Latency: Time to first token of tokens received, in seconds, after API request sent.

Price: Price per token, represented as USD per million Tokens. Price is a blend of Input & Output token prices (3:1 ratio).

Output price: Price per token generated by the model (received from the API), represented as USD per million Tokens.

Input price: Price per token included in the request/message sent to the API, represented as USD per million Tokens.

Time period: Metrics are 'live' and are based on the past 14 days of measurements, measurements are taken 8 times a day for single requests and 2 times per day for parallel requests.