LLM API Providers Leaderboard - Comparison of over 100 LLM endpoints

Comparison and ranking of API provider performance for over 100 AI LLM Model endpoints across performance key metrics including price, output speed, latency, context window & others. For more details including relating to our methodology, see our FAQs.

API providers compared: OpenAI, Reka AI, Mistral, Microsoft Azure, Amazon Bedrock, Groq, Together.ai, Anthropic, Perplexity, Google, Fireworks, Baseten, Cohere, Lepton AI, Speechmatics, Deepinfra, Replicate, Runpod, Rev AI, DeepSeek, fal.ai, AssemblyAI, Deepgram, Gladia, Stability.ai, Midjourney, Databricks, ElevenLabs, IBM, OctoAI, Cartesia, LMNT, and AI21 Labs.

Context
Model Quality
Price
Output tokens/s
Latency
Further
Analysis
OpenAI logo
OpenAI logoGPT-4o
128k
100
$7.50
72.1
0.49
OpenAI logo
OpenAI logoGPT-4 Turbo
128k
94
$15.00
27.0
0.71
Microsoft Azure logo
OpenAI logoGPT-4 Turbo
128k
94
$15.00
28.0
0.55
OpenAI logo
OpenAI logoGPT-4
8k
93
$37.50
21.0
0.86
Microsoft Azure logo
OpenAI logoGPT-4
8k
93
$37.50
20.4
0.55
OpenAI logo
OpenAI logoGPT-3.5 Turbo
16k
65
$0.75
69.0
0.44
Microsoft Azure logo
OpenAI logoGPT-3.5 Turbo
16k
65
$0.75
57.4
0.32
OpenAI logo
OpenAI logoGPT-3.5 Turbo Instruct
4k
60
$1.63
73.8
0.36
Microsoft Azure logo
OpenAI logoGPT-3.5 Turbo Instruct
4k
60
$1.63
133.0
0.61
Google logo
Google logoGemini 1.5 Flash
1m
83
$0.53
139.5
1.34
Google logo
Google logoGemini 1.5 Pro
1m
93
$5.25
63.4
1.49
Google logo
Google logoGemini 1.0 Pro
33k
62
$0.75
85.9
2.43
Fireworks logo
Google logoGemma 7B
8k
57
$0.20
234.1
0.26
Deepinfra logo
Google logoGemma 7B
8k
57
$0.07
64.4
0.29
Groq logo
Google logoGemma 7B
8k
57
$0.07
1,011.0
0.91
Together.ai logo
Google logoGemma 7B
8k
57
$0.20
140.9
0.39
Replicate logo
Meta logoLlama 3 (70B)
8k
88
$1.18
49.9
2.00
Amazon Bedrock logo
Meta logoLlama 3 (70B)
8k
88
$2.86
48.1
0.50
OctoAI logo
Meta logoLlama 3 (70B)
8k
88
$0.90
61.3
0.29
Microsoft Azure logo
Meta logoLlama 3 (70B)
8k
88
$5.67
18.6
2.66
Fireworks logo
Meta logoLlama 3 (70B)
8k
88
$0.90
118.9
0.27
Deepinfra logo
Meta logoLlama 3 (70B)
8k
88
$0.64
21.1
0.36
Groq logo
Meta logoLlama 3 (70B)
8k
88
$0.64
356.1
0.42
Perplexity logo
Meta logoLlama 3 (70B)
8k
88
$1.00
49.6
0.33
Together.ai logo
Meta logoLlama 3 (70B)
8k
88
$0.90
119.2
0.73
Replicate logo
Meta logoLlama 3 (8B)
8k
65
$0.10
80.6
1.74
Amazon Bedrock logo
Meta logoLlama 3 (8B)
8k
65
$0.45
78.4
0.31
OctoAI logo
Meta logoLlama 3 (8B)
8k
65
$0.15
131.8
0.21
Microsoft Azure logo
Meta logoLlama 3 (8B)
8k
65
$0.55
76.5
0.89
Fireworks logo
Meta logoLlama 3 (8B)
8k
65
$0.20
242.2
0.25
Deepinfra logo
Meta logoLlama 3 (8B)
8k
65
$0.08
115.4
0.19
Groq logo
Meta logoLlama 3 (8B)
8k
65
$0.06
1,211.3
0.40
Perplexity logo
Meta logoLlama 3 (8B)
8k
65
$0.20
119.8
0.26
Together.ai logo
Meta logoLlama 3 (8B)
8k
65
$0.20
268.8
0.38
Fireworks logo
Meta logoCode Llama (70B)
4k
58
$0.90
Deepinfra logo
Meta logoCode Llama (70B)
4k
58
$0.60
33.3
0.45
Perplexity logo
Meta logoCode Llama (70B)
16k
58
$1.00
Together.ai logo
Meta logoCode Llama (70B)
4k
58
$0.90
29.8
0.55
Replicate logo
Meta logoLlama 2 Chat (70B)
4k
50
$1.18
52.6
1.94
Amazon Bedrock logo
Meta logoLlama 2 Chat (70B)
4k
50
$2.10
43.5
0.52
OctoAI logo
Meta logoLlama 2 Chat (70B)
4k
50
$0.90
131.3
0.21
Microsoft Azure logo
Meta logoLlama 2 Chat (70B)
4k
50
$1.60
17.5
3.25
Fireworks logo
Meta logoLlama 2 Chat (70B)
4k
50
$0.90
92.7
0.33
Deepinfra logo
Meta logoLlama 2 Chat (70B)
4k
50
$0.76
98.6
0.22
Perplexity logo
Meta logoLlama 2 Chat (70B)
4k
50
$1.00
Together.ai logo
Meta logoLlama 2 Chat (70B)
4k
50
$0.90
33.9
0.82
AI21 Labs logo
AI21 Labs logoJamba Instruct
256k
63
$0.55
66.5
0.44
Replicate logo
Meta logoLlama 2 Chat (13B)
4k
36
$0.20
75.9
1.46
Amazon Bedrock logo
Meta logoLlama 2 Chat (13B)
4k
36
$0.81
50.7
0.37
OctoAI logo
Meta logoLlama 2 Chat (13B)
4k
36
$0.20
131.0
0.21
Microsoft Azure logo
Meta logoLlama 2 Chat (13B)
4k
36
$0.84
43.9
1.51
Fireworks logo
Meta logoLlama 2 Chat (13B)
4k
36
$0.20
106.8
0.31
Deepinfra logo
Meta logoLlama 2 Chat (13B)
4k
36
$0.35
99.3
0.22
Together.ai logo
Meta logoLlama 2 Chat (13B)
4k
36
$0.30
44.5
0.44
Replicate logo
Meta logoLlama 2 Chat (7B)
4k
27
$0.10
148.9
1.22
Microsoft Azure logo
Meta logoLlama 2 Chat (7B)
4k
27
$0.56
72.4
1.03
Fireworks logo
Meta logoLlama 2 Chat (7B)
4k
27
$0.20
166.4
0.27
Deepinfra logo
Meta logoLlama 2 Chat (7B)
4k
27
$0.20
23.7
0.42
Together.ai logo
Meta logoLlama 2 Chat (7B)
4k
27
$0.20
92.1
0.44
Mistral logo
Mistral logoMixtral 8x22B
65k
78
$3.00
67.7
0.25
OctoAI logo
Mistral logoMixtral 8x22B
65k
78
$1.20
84.6
0.28
Fireworks logo
Mistral logoMixtral 8x22B
65k
78
$1.20
81.5
0.25
Deepinfra logo
Mistral logoMixtral 8x22B
65k
78
$0.65
35.6
0.26
Perplexity logo
Mistral logoMixtral 8x22B
16k
78
$1.00
Together.ai logo
Mistral logoMixtral 8x22B
65k
78
$1.20
62.9
0.65
Mistral logo
Mistral logoMistral Large
33k
75
$6.00
38.6
0.24
Amazon Bedrock logo
Mistral logoMistral Large
33k
75
$12.00
34.5
0.43
Microsoft Azure logo
Mistral logoMistral Large
33k
75
$6.00
25.0
2.07
Mistral logo
Mistral logoMistral Medium
33k
73
$4.05
37.4
0.34
Mistral logo
Mistral logoMistral Small
33k
71
$1.50
36.5
0.26
Microsoft Azure logo
Mistral logoMistral Small
33k
71
$1.50
62.4
1.27
Mistral logo
Mistral logoMixtral 8x7B
33k
65
$0.70
70.3
0.26
Replicate logo
Mistral logoMixtral 8x7B
33k
65
$0.47
107.2
1.61
Amazon Bedrock logo
Mistral logoMixtral 8x7B
33k
65
$0.51
63.9
0.37
OctoAI logo
Mistral logoMixtral 8x7B
33k
65
$0.45
78.5
0.27
Lepton AI logo
Mistral logoMixtral 8x7B
33k
65
$0.50
103.2
0.28
Fireworks logo
Mistral logoMixtral 8x7B
33k
65
$0.50
261.4
0.25
Deepinfra logo
Mistral logoMixtral 8x7B
33k
65
$0.24
65.2
0.21
Groq logo
Mistral logoMixtral 8x7B
33k
65
$0.24
552.1
0.44
Perplexity logo
Mistral logoMixtral 8x7B
16k
65
$0.60
111.5
0.26
Together.ai logo
Mistral logoMixtral 8x7B
33k
65
$0.60
83.9
0.37
Mistral logo
Mistral logoMistral 7B
33k
39
$0.25
78.8
0.27
Replicate logo
Mistral logoMistral 7B
33k
39
$0.10
81.5
1.60
Amazon Bedrock logo
Mistral logoMistral 7B
33k
39
$0.16
72.7
0.34
OctoAI logo
Mistral logoMistral 7B
33k
39
$0.15
146.9
0.21
Fireworks logo
Mistral logoMistral 7B
33k
39
$0.20
258.7
0.18
Deepinfra logo
Mistral logoMistral 7B
33k
39
$0.07
71.9
0.30
Perplexity logo
Mistral logoMistral 7B
16k
39
$0.20
104.1
0.27
Together.ai logo
Mistral logoMistral 7B
8k
39
$0.20
64.0
0.38
Baseten logo
Mistral logoMistral 7B
4k
39
$0.20
217.0
0.18
Anthropic logo
Anthropic logoClaude 3.5 Sonnet
200k
100
$6.00
79.4
0.86
Amazon Bedrock logo
Anthropic logoClaude 3 Opus
200k
94
$30.00
22.1
1.83
Anthropic logo
Anthropic logoClaude 3 Opus
200k
94
$30.00
24.7
2.09
Amazon Bedrock logo
Anthropic logoClaude 3 Sonnet
200k
78
$6.00
43.8
0.88
Google logo
Anthropic logoClaude 3 Sonnet
200k
78
$6.00
Anthropic logo
Anthropic logoClaude 3 Sonnet
200k
78
$6.00
59.9
1.00
Amazon Bedrock logo
Anthropic logoClaude 3 Haiku
200k
72
$0.50
93.4
0.45
Google logo
Anthropic logoClaude 3 Haiku
200k
72
$0.50
Anthropic logo
Anthropic logoClaude 3 Haiku
200k
72
$0.50
145.2
0.56
Together.ai logo
Alibaba logoQwen2 (72B)
128k
$0.90
44.4
0.51
Anthropic logo
Anthropic logoClaude 2.0
100k
69
$12.00
38.4
1.28
Amazon Bedrock logo
Anthropic logoClaude 2.1
200k
63
$12.00
35.1
1.69
Anthropic logo
Anthropic logoClaude 2.1
200k
63
$12.00
37.1
1.22
Amazon Bedrock logo
Anthropic logoClaude Instant
100k
63
$1.20
80.1
0.56
Anthropic logo
Anthropic logoClaude Instant
100k
63
$1.20
98.4
0.57
Amazon Bedrock logo
Cohere logoCommand Light
4k
$0.38
33.9
0.57
Cohere logo
Cohere logoCommand Light
4k
$0.38
67.6
0.26
Amazon Bedrock logo
Cohere logoCommand
4k
$1.63
23.2
0.58
Cohere logo
Cohere logoCommand
4k
$1.25
24.6
0.57
Cohere logo
Cohere logoCommand-R+
128k
74
$6.00
63.8
0.31
Microsoft Azure logo
Cohere logoCommand-R+
128k
74
$6.00
59.0
0.47
Cohere logo
Cohere logoCommand-R
128k
62
$0.75
146.8
0.21
Microsoft Azure logo
Cohere logoCommand-R
128k
62
$0.75
48.5
0.46
Deepinfra logo
OpenChat logoOpenChat 3.5
8k
54
$0.07
65.7
0.31
Together.ai logo
OpenChat logoOpenChat 3.5
8k
54
$0.20
72.9
0.35
Lepton AI logo
Databricks logoDBRX
33k
74
$0.90
Fireworks logo
Databricks logoDBRX
33k
74
$1.20
53.9
0.36
Databricks logo
Databricks logoDBRX
33k
74
$3.38
91.1
0.63
Together.ai logo
Databricks logoDBRX
33k
74
$1.20
74.7
0.48
DeepSeek logo
DeepSeek logoDeepSeek-V2
128k
82
$0.17
16.9
1.63
Together.ai logo
Snowflake logoArctic
4k
63
$2.40
72.6
0.41

Key definitions

Quality: Index represents normalized average relative performance across Chatbot arena, MMLU & MT-Bench.
Context window: Maximum number of combined input & output tokens. Output tokens commonly have a significantly lower limit (varied by model).
Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API).
Latency: Time to first token of tokens received, in seconds, after API request sent.
Price: Price per token, represented as USD per million Tokens. Price is a blend of Input & Output token prices (3:1 ratio).
Output price: Price per token generated by the model (received from the API), represented as USD per million Tokens.
Input price: Price per token included in the request/message sent to the API, represented as USD per million Tokens.
Time period: Metrics are 'live' and are based on the past 14 days of measurements, measurements are taken 8 times a day for single requests and 2 times per day for parallel requests.