LLM API Providers Leaderboard - Comparison of over 100 LLM endpoints

Comparison and ranking of API provider performance for over 100 AI LLM Model endpoints across performance key metrics including price, output speed, latency, context window & others. For more details including relating to our methodology, see our FAQs.

API providers compared: OpenAI, Playground AI, Mistral, Microsoft Azure, Ideogram, Amazon Bedrock, Groq, Together.ai, Anthropic, Perplexity, Google, Fireworks, Cerebras, Cohere, Lepton AI, Speechmatics, Deepinfra, Replicate, Runpod, Rev AI, fal.ai, AssemblyAI, DeepSeek, Reka AI, Deepgram, Gladia, Stability.ai, Baseten, Midjourney, Databricks, ElevenLabs, IBM, SambaNova, OctoAI, Cartesia, LMNT, 01.AI, and AI21 Labs.

Context
Model Quality
Price
Output tokens/s
Latency
Further
Analysis
OpenAI logo
OpenAI logo
o1-preview
128k
$26.25
23.5
43.63
OpenAI logo
OpenAI logo
o1-mini
128k
$5.25
73.9
13.98
OpenAI logo
OpenAI logo
GPT-4o (Aug 6)
128k
100
$4.38
92.6
0.40
OpenAI logo
OpenAI logo
GPT-4o
128k
100
$7.50
110.0
0.36
Microsoft Azure logo
OpenAI logo
GPT-4o
128k
100
$7.50
108.1
0.35
OpenAI logo
OpenAI logo
GPT-4o mini
128k
88
$0.26
140.1
0.39
Replicate logo
Meta logo
Llama 3.1 405B
128k
100
$9.50
18.4
0.27
Amazon Bedrock logo
Meta logo
Llama 3.1 405B
128k
100
$7.99
13.3
1.83
OctoAI logo
Meta logo
Llama 3.1 405B
128k
100
$4.50
30.0
0.47
Lepton AI logo
Meta logo
Llama 3.1 405B
128k
100
$2.80
27.7
1.00
Microsoft Azure logo
Meta logo
Llama 3.1 405B
128k
100
$8.00
14.2
0.61
Fireworks logo
Meta logo
Llama 3.1 405B
128k
100
$3.00
72.7
0.66
Deepinfra logo
Meta logo
Llama 3.1 405B
33k
100
$1.79
22.7
0.46
SambaNova logo
Meta logo
Llama 3.1 405B
8k
100
$6.25
129.3
1.46
Databricks logo
Meta logo
Llama 3.1 405B
128k
100
$7.50
27.6
0.66
Together.ai Turbo logo
Meta logo
Llama 3.1 405B Turbo
8k
100
$5.00
88.0
0.58
Cerebras logo
Meta logo
Llama 3.1 70B
8k
95
$0.60
481.5
0.25
Amazon Bedrock logo
Meta logo
Llama 3.1 70B
128k
95
$0.99
31.8
0.72
OctoAI logo
Meta logo
Llama 3.1 70B
128k
95
$0.90
53.6
0.28
Lepton AI logo
Meta logo
Llama 3.1 70B
128k
95
$0.80
55.8
0.63
Microsoft Azure logo
Meta logo
Llama 3.1 70B
128k
95
$2.90
28.5
0.56
Fireworks logo
Meta logo
Llama 3.1 70B
128k
95
$0.90
73.1
0.41
Deepinfra logo
Meta logo
Llama 3.1 70B
128k
95
$0.36
27.1
0.30
Groq logo
Meta logo
Llama 3.1 70B
128k
95
$0.64
249.5
0.45
SambaNova logo
Meta logo
Llama 3.1 70B
8k
95
$0.75
424.8
0.84
Databricks logo
Meta logo
Llama 3.1 70B
128k
95
$1.50
45.4
0.58
Perplexity logo
Meta logo
Llama 3.1 70B
128k
95
$1.00
52.2
0.23
Together.ai Turbo logo
Meta logo
Llama 3.1 70B Turbo
128k
95
$0.88
84.8
0.47
Cerebras logo
Meta logo
Llama 3.1 8B
8k
66
$0.10
1,929.1
0.28
Amazon Bedrock logo
Meta logo
Llama 3.1 8B
128k
66
$0.22
92.6
0.41
OctoAI logo
Meta logo
Llama 3.1 8B
128k
66
$0.15
165.1
0.24
Lepton AI logo
Meta logo
Llama 3.1 8B
128k
66
$0.07
204.3
0.43
Microsoft Azure logo
Meta logo
Llama 3.1 8B
128k
66
$0.38
71.4
0.40
Fireworks logo
Meta logo
Llama 3.1 8B
128k
66
$0.20
288.9
0.27
Deepinfra logo
Meta logo
Llama 3.1 8B
128k
66
$0.06
86.3
0.21
Groq logo
Meta logo
Llama 3.1 8B
128k
66
$0.06
751.2
0.38
SambaNova logo
Meta logo
Llama 3.1 8B
8k
66
$0.13
1,095.2
0.44
Perplexity logo
Meta logo
Llama 3.1 8B
128k
66
$0.20
161.4
0.17
Together.ai Turbo logo
Meta logo
Llama 3.1 8B Turbo
128k
66
$0.18
200.0
0.39
Google (Vertex) logo
Google logo
Gemini 1.5 Pro (Vertex)
2m
95
$5.25
Google (AI Studio) logo
Google logo
Gemini 1.5 Pro (AI Studio)
2m
95
$5.25
63.8
0.89
Google (Vertex) logo
Google logo
Gemini 1.5 Flash (Vertex)
1m
84
$0.13
Google (AI Studio) logo
Google logo
Gemini 1.5 Flash (AI Studio)
1m
84
$0.13
209.8
0.40
Together.ai logo
Google logo
Gemma 2 27B
8k
78
$0.80
70.8
0.34
Fireworks logo
Google logo
Gemma 2 9B
8k
71
$0.20
133.7
0.32
Deepinfra logo
Google logo
Gemma 2 9B
8k
71
$0.06
67.3
0.32
Groq logo
Google logo
Gemma 2 9B
8k
71
$0.20
673.3
0.20
Together.ai logo
Google logo
Gemma 2 9B
8k
71
$0.30
120.0
0.40
Amazon Bedrock logo
Anthropic logo
Claude 3.5 Sonnet
200k
98
$6.00
52.4
0.97
Anthropic logo
Anthropic logo
Claude 3.5 Sonnet
200k
98
$6.00
94.9
1.02
Amazon Bedrock logo
Anthropic logo
Claude 3 Opus
200k
93
$30.00
23.3
1.71
Anthropic logo
Anthropic logo
Claude 3 Opus
200k
93
$30.00
27.3
1.85
Amazon Bedrock logo
Anthropic logo
Claude 3 Haiku
200k
74
$0.50
118.1
0.44
Anthropic logo
Anthropic logo
Claude 3 Haiku
200k
74
$0.50
157.7
0.58
Mistral logo
Mistral logo
Mistral Large 2
128k
91
$4.50
39.4
0.51
Amazon Bedrock logo
Mistral logo
Mistral Large 2
128k
91
$4.50
42.1
0.43
Mistral logo
Mistral logo
Mixtral 8x22B
65k
71
$3.00
66.3
0.47
OctoAI logo
Mistral logo
Mixtral 8x22B
65k
71
$1.20
39.2
0.32
Fireworks logo
Mistral logo
Mixtral 8x22B
65k
71
$1.20
50.6
0.38
Deepinfra logo
Mistral logo
Mixtral 8x22B
65k
71
$0.65
55.0
0.26
Together.ai logo
Mistral logo
Mixtral 8x22B
65k
71
$1.20
72.1
0.39
Mistral logo
Mistral logo
Mistral NeMo
128k
64
$0.30
136.9
0.39
OctoAI logo
Mistral logo
Mistral NeMo
128k
64
$0.20
158.5
0.31
Deepinfra logo
Mistral logo
Mistral NeMo
128k
64
$0.13
63.4
0.23
Mistral logo
Mistral logo
Mistral Small
33k
71
$1.50
49.8
0.42
Microsoft Azure logo
Mistral logo
Mistral Small
33k
71
$1.50
51.7
0.39
Mistral logo
Mistral logo
Mixtral 8x7B
33k
61
$0.70
87.0
0.41
Replicate logo
Mistral logo
Mixtral 8x7B
33k
61
$0.47
43.9
0.27
Amazon Bedrock logo
Mistral logo
Mixtral 8x7B
33k
61
$0.51
60.8
0.40
OctoAI logo
Mistral logo
Mixtral 8x7B
33k
61
$0.45
77.9
0.29
Lepton AI logo
Mistral logo
Mixtral 8x7B
33k
61
$0.50
36.9
0.76
Fireworks logo
Mistral logo
Mixtral 8x7B
33k
61
$0.50
94.9
0.33
Deepinfra logo
Mistral logo
Mixtral 8x7B
33k
61
$0.24
46.2
0.31
Groq logo
Mistral logo
Mixtral 8x7B
33k
61
$0.24
543.0
0.22
Databricks logo
Mistral logo
Mixtral 8x7B
33k
61
$0.63
86.6
0.50
Together.ai logo
Mistral logo
Mixtral 8x7B
33k
61
$0.60
109.3
0.36
Mistral logo
Mistral logo
Codestral-Mamba
256k
$0.25
95.7
0.49
Amazon Bedrock logo
Cohere logo
Command-R+ (08-2024)
128k
$6.00
45.7
0.56
Cohere logo
Cohere logo
Command-R+ (08-2024)
128k
$4.38
65.9
0.31
Amazon Bedrock logo
Cohere logo
Command-R (08-2024)
128k
$0.75
103.2
0.39
Cohere logo
Cohere logo
Command-R (08-2024)
128k
$0.26
113.5
0.22
Amazon Bedrock logo
Cohere logo
Command-R+ (04-2024)
128k
75
$6.00
45.5
0.57
Cohere logo
Cohere logo
Command-R+ (04-2024)
128k
75
$6.00
64.0
0.31
Microsoft Azure logo
Cohere logo
Command-R+ (04-2024)
128k
75
$6.00
66.2
0.52
Amazon Bedrock logo
Cohere logo
Command-R (03-2024)
128k
63
$0.75
103.1
0.39
Cohere logo
Cohere logo
Command-R (03-2024)
128k
63
$0.75
152.5
0.21
Microsoft Azure logo
Cohere logo
Command-R (03-2024)
128k
63
$0.75
103.7
0.47
Perplexity logo
Perplexity logo
Sonar Large
33k
$1.00
41.7
0.25
Perplexity logo
Perplexity logo
Sonar Small
33k
$0.20
138.8
0.17
Perplexity logo
Perplexity logo
Sonar 3.1 Small
131k
$0.20
138.0
0.17
Perplexity logo
Perplexity logo
Sonar 3.1 Large
131k
$1.00
67.8
0.21
Microsoft Azure logo
Microsoft Azure logo
Phi-3 Medium 14B
128k
$0.75
49.7
0.45
Deepinfra logo
Microsoft Azure logo
Phi-3 Medium 14B
4k
$0.14
71.7
0.23
Databricks logo
Databricks logo
DBRX
33k
62
$1.13
85.0
0.49
Together.ai logo
Databricks logo
DBRX
33k
62
$1.20
104.4
0.34
Reka AI logo
Reka AI logo
Reka Core
128k
90
$6.00
14.3
1.13
Reka AI logo
Reka AI logo
Reka Flash
128k
78
$1.10
29.8
0.96
Reka AI logo
Reka AI logo
Reka Edge
64k
60
$0.55
34.1
0.97
AI21 Labs logo
AI21 Labs logo
Jamba 1.5 Large
256k
86
$3.50
62.4
1.19
AI21 Labs logo
AI21 Labs logo
Jamba 1.5 Mini
256k
64
$0.25
158.4
1.03
DeepSeek logo
DeepSeek logo
DeepSeek-Coder-V2
128k
$0.17
17.1
1.21
DeepSeek logo
DeepSeek logo
DeepSeek-V2.5
128k
$0.17
16.0
1.21
DeepSeek logo
DeepSeek logo
DeepSeek-V2
128k
82
$0.17
17.7
1.22
Deepinfra logo
Alibaba logo
Qwen2 72B
33k
83
$0.36
36.4
0.29
Together.ai logo
Alibaba logo
Qwen2 72B
33k
83
$0.90
54.6
0.36
Fireworks logo
01.AI logo
Yi-Large
32k
81
$3.00
65.5
0.62
OpenAI logo
OpenAI logo
GPT-4 Turbo
128k
94
$15.00
31.4
0.63
Microsoft Azure logo
OpenAI logo
GPT-4 Turbo
128k
94
$15.00
49.0
0.53
OpenAI logo
OpenAI logo
GPT-3.5 Turbo
16k
59
$0.75
82.1
0.42
Microsoft Azure logo
OpenAI logo
GPT-3.5 Turbo
16k
59
$0.75
75.8
0.34
OpenAI logo
OpenAI logo
GPT-3.5 Turbo Instruct
4k
60
$1.63
88.9
0.40
Microsoft Azure logo
OpenAI logo
GPT-3.5 Turbo Instruct
4k
60
$1.63
137.8
0.60
OpenAI logo
OpenAI logo
GPT-4
8k
84
$37.50
20.4
0.66
Microsoft Azure logo
OpenAI logo
GPT-4
8k
84
$37.50
41.4
0.54
Replicate logo
Meta logo
Llama 3 70B
8k
83
$1.18
47.1
0.27
Amazon Bedrock logo
Meta logo
Llama 3 70B
8k
83
$2.86
51.5
0.46
OctoAI logo
Meta logo
Llama 3 70B
8k
83
$0.90
63.1
0.33
Lepton AI logo
Meta logo
Llama 3 70B
8k
83
$0.80
29.5
0.89
Microsoft Azure logo
Meta logo
Llama 3 70B
8k
83
$2.90
18.9
0.74
Fireworks logo
Meta logo
Llama 3 70B
8k
83
$0.90
108.1
0.36
Deepinfra logo
Meta logo
Llama 3 70B
8k
83
$0.36
18.4
0.32
Groq logo
Meta logo
Llama 3 70B
8k
83
$0.64
317.7
0.22
Together.ai (Reference, FP16) logo
Meta logo
Llama 3 70B (Reference, FP16)
8k
83
$0.90
154.2
0.41
Together.ai (Turbo, FP8) logo
Meta logo
Llama 3 70B (Turbo, FP8)
8k
83
$0.88
89.5
0.49
Replicate logo
Meta logo
Llama 3 8B
8k
64
$0.10
75.8
0.27
Amazon Bedrock logo
Meta logo
Llama 3 8B
8k
64
$0.38
78.2
0.34
Lepton AI logo
Meta logo
Llama 3 8B
8k
64
$0.07
71.6
5.31
Microsoft Azure logo
Meta logo
Llama 3 8B
8k
64
$0.38
73.0
0.41
Fireworks logo
Meta logo
Llama 3 8B
8k
64
$0.20
114.0
0.36
Deepinfra logo
Meta logo
Llama 3 8B
8k
64
$0.06
111.3
0.20
Groq logo
Meta logo
Llama 3 8B
8k
64
$0.06
1,201.3
0.31
Together.ai logo
Meta logo
Llama 3 8B
8k
64
$0.20
300.5
0.42
Replicate logo
Meta logo
Llama 2 Chat 70B
4k
57
$1.18
46.7
0.31
Amazon Bedrock logo
Meta logo
Llama 2 Chat 70B
4k
57
$2.10
35.6
0.47
OctoAI logo
Meta logo
Llama 2 Chat 70B
4k
57
$0.90
174.1
0.23
Microsoft Azure logo
Meta logo
Llama 2 Chat 70B
4k
57
$1.60
17.2
0.95
Amazon Bedrock logo
Meta logo
Llama 2 Chat 13B
4k
39
$0.81
52.6
0.40
OctoAI logo
Meta logo
Llama 2 Chat 13B
4k
39
$0.20
172.8
0.23
Together.ai logo
Meta logo
Llama 2 Chat 13B
4k
39
$0.30
52.5
0.48
Replicate logo
Meta logo
Llama 2 Chat 7B
4k
29
$0.10
122.6
0.27
Microsoft Azure logo
Meta logo
Llama 2 Chat 7B
4k
29
$0.56
68.6
0.54
Groq logo
Google logo
Gemma 7B
8k
45
$0.07
1,023.3
0.83
Google (AI Studio) logo
Google logo
Gemini 1.0 Pro (AI Studio)
33k
62
$0.75
99.8
1.15
Amazon Bedrock logo
Anthropic logo
Claude 3 Sonnet
200k
80
$6.00
52.2
0.81
Anthropic logo
Anthropic logo
Claude 3 Sonnet
200k
80
$6.00
59.6
0.94
Amazon Bedrock logo
Anthropic logo
Claude 2.1
200k
55
$12.00
27.7
1.82
Anthropic logo
Anthropic logo
Claude 2.1
200k
55
$12.00
30.9
1.09
Anthropic logo
Anthropic logo
Claude 2.0
100k
70
$12.00
31.2
1.10
Amazon Bedrock logo
Anthropic logo
Claude Instant
100k
63
$1.20
61.7
0.59
Anthropic logo
Anthropic logo
Claude Instant
100k
63
$1.20
100.9
0.54
Mistral logo
Mistral logo
Mistral Large
33k
76
$6.00
40.4
0.51
Amazon Bedrock logo
Mistral logo
Mistral Large
33k
76
$6.00
Microsoft Azure logo
Mistral logo
Mistral Large
33k
76
$6.00
22.8
0.38
Mistral logo
Mistral logo
Mistral 7B
33k
40
$0.25
99.4
0.41
Replicate logo
Mistral logo
Mistral 7B
33k
40
$0.10
38.5
0.29
Amazon Bedrock logo
Mistral logo
Mistral 7B
33k
40
$0.16
75.8
0.36
OctoAI logo
Mistral logo
Mistral 7B
33k
40
$0.15
168.5
0.21
Lepton AI logo
Mistral logo
Mistral 7B
33k
40
$0.07
95.6
1.11
Deepinfra logo
Mistral logo
Mistral 7B
33k
40
$0.06
107.0
0.21
Perplexity logo
Mistral logo
Mistral 7B
16k
40
$0.20
125.6
0.19
Together.ai logo
Mistral logo
Mistral 7B
8k
40
$0.20
127.9
0.30
Mistral logo
Mistral logo
Codestral
33k
$1.50
50.0
0.42
Mistral logo
Mistral logo
Mistral Medium
33k
70
$4.09
38.4
0.75
Amazon Bedrock logo
Cohere logo
Command
4k
$1.63
21.8
0.56
Cohere logo
Cohere logo
Command
4k
$1.25
22.3
0.39
Amazon Bedrock logo
Cohere logo
Command Light
4k
$0.38
32.8
0.60
Cohere logo
Cohere logo
Command Light
4k
$0.38
59.4
0.25
Deepinfra logo
OpenChat logo
OpenChat 3.5
8k
50
$0.06
70.6
0.32
AI21 Labs logo
AI21 Labs logo
Jamba Instruct
256k
63
$0.55
80.7
0.91
Microsoft Azure logo
AI21 Labs logo
Jamba Instruct
256k
63
$0.55
61.6
0.32

Key definitions

Artificial Analysis Quality Index: Average result across our evaluations covering different dimensions of model intelligence. Currently includes MMLU, GPQA, Math & HumanEval. OpenAI o1 model figures are preliminary and are based on figures stated by OpenAI. See methodology for more details.
Context window: Maximum number of combined input & output tokens. Output tokens commonly have a significantly lower limit (varied by model).
Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).
Latency: Time to first token of tokens received, in seconds, after API request sent. For models which do not support streaming, this represents time to receive the completion.
Price: Price per token, represented as USD per million Tokens. Price is a blend of Input & Output token prices (3:1 ratio).
Output price: Price per token generated by the model (received from the API), represented as USD per million Tokens.
Input price: Price per token included in the request/message sent to the API, represented as USD per million Tokens.
Time period: Metrics are 'live' and are based on the past 14 days of measurements, measurements are taken 8 times a day for single requests and 2 times per day for parallel requests.