Llama 3.3 Instruct 70B API Provider Benchmarking & Analysis

Analysis of API providers for Llama 3.3 Instruct 70B across performance metrics including latency (time to first token), output speed (output tokens per second), price and others. API providers benchmarked include SambaNova, Amazon Bedrock, Snowflake Snowflake, Groq, Nebius Base, Fireworks, FriendliAI, Hyperbolic, Google Vertex, Weights & Biases, CompactifAI, Databricks, Together.ai Turbo, Lightning AI, Microsoft Azure, Scaleway, DeepInfra (Turbo, FP8), Novita, Cloudflare, Parasail (FP8).

Fastest

Groq

493.6 t/s

SambaNova

435.4 t/s

CompactifAI

324.6 t/s

Amazon

192.8 t/s

Azure

189.2 t/s

Output speed

Total 20 providers

Lowest Latency

Google Vertex

0.23 s

Lightning AI

0.25 s

Groq

0.27 s

CompactifAI

0.28 s

FriendliAI

0.37 s

Time to first answer token

Total 20 providers

Lowest Price

DeepInfra (Turbo, FP8)

$0.16

Nebius Base

$0.20

Novita

$0.20

Parasail (FP8)

$0.29

Lightning AI

$0.30

Blended price (per 1M tokens, 3:1 Input-Output ratio)

Total 20 providers

Llama 3.3 70B is available through 20 API providers, each offering different performance characteristics and pricing. Below is a comparison of the key metrics across providers.

For output speed, the top providers are Groq (493.6 t/s), SambaNova (435.4 t/s), CompactifAI (324.6 t/s). Speed varies significantly across providers, with a 161% difference between the fastest and slowest.
For latency, Google Vertex (0.23s), Lightning AI (0.25s), Groq (0.27s) offer the lowest time to first token.
For pricing, DeepInfra (Turbo, FP8) (0.16), Nebius Base (0.20), Novita (0.20) offer the lowest blended prices per 1M tokens.

Highlights

Speed

Output tokens per second · Higher is better

End-to-End Response Time

Seconds · Lower is better

Price

USD per 1M tokens (3:1 input-output ratio) · Lower is better

Update: Default performance benchmarking workload has updated to 10k input tokens to better reflect production use cases. You can still select different workloads above.

Pricing

Pricing: Cache Hit, Input, and Output

USD per 1M tokens · Lower is better · 1,000 Input Tokens (Coding workload)

Price per token for cached prompts (previously processed), typically offering a significant discount compared to regular input price, represented as USD per million tokens. The values shown here are the cache hit price; cache write and cache storage are billed separately and vary by provider — see "Cache pricing by provider" for detail.

Price per token included in the request/message sent to the API, represented as USD per million Tokens.

The blended bar shown here uses cache hit price only. Other caching costs differ by provider:

Anthropic: charges a separate cache write fee, with different rates for 5-minute and 1-hour TTLs (1-hour TTL is more expensive). Blended price charts use Anthropic cache write price for the input leg.
Google (Vertex/Gemini): charges a per-hour cache storage fee in addition to cache hit pricing. Some providers also use tiered pricing for prompts above 200K tokens.
OpenAI, DeepSeek, others: typically charge only cache hit pricing with no write or storage fee.

See Prompt Caching for the full breakdown.

Price per token generated by the model (received from the API), represented as USD per million Tokens.

Pricing: Blended Price

Blended at 7:2:1 (Cache Hit : Input : Output) · USD per 1M tokens · Lower is better

Price per token, represented as USD per million Tokens. Price is a blend of cache hit, input, and output token prices using the selected ratio.

The blended bar shown here uses cache hit price only. Other caching costs differ by provider:

Anthropic: charges a separate cache write fee, with different rates for 5-minute and 1-hour TTLs (1-hour TTL is more expensive). Blended price charts use Anthropic cache write price for the input leg.
Google (Vertex/Gemini): charges a per-hour cache storage fee in addition to cache hit pricing. Some providers also use tiered pricing for prompts above 200K tokens.
OpenAI, DeepSeek, others: typically charge only cache hit pricing with no write or storage fee.

See Prompt Caching for the full breakdown.

Figures represent median (P50) measurement over the past 72 hours to reflect sustained changes in performance.

Speed vs. Price

Blended at 7:2:1 (Cache Hit : Input : Output) · Output speed: output tokens per second · Price: USD per 1M tokens

Most attractive quadrant

Amazon

Azure

Cloudflare

CompactifAI

Databricks

DeepInfra (Turbo, FP8)

Fireworks

FriendliAI

Google Vertex

Groq

Hyperbolic

Lightning AI

Nebius Base

Novita

Parasail (FP8)

SambaNova

Scaleway

Snowflake

Together.ai Turbo

Weights & Biases

Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).

Price per token, represented as USD per million Tokens. Price is a blend of cache hit, input, and output token prices using the selected ratio.

The blended bar shown here uses cache hit price only. Other caching costs differ by provider:

Anthropic: charges a separate cache write fee, with different rates for 5-minute and 1-hour TTLs (1-hour TTL is more expensive). Blended price charts use Anthropic cache write price for the input leg.
Google (Vertex/Gemini): charges a per-hour cache storage fee in addition to cache hit pricing. Some providers also use tiered pricing for prompts above 200K tokens.
OpenAI, DeepSeek, others: typically charge only cache hit pricing with no write or storage fee.

See Prompt Caching for the full breakdown.

Figures represent median (P50) measurement over the past 72 hours to reflect sustained changes in performance.

Speed

Measured by Output Speed (tokens per second)

Output Speed: Llama 3.3 70B Providers

Output tokens per second · Higher is better · 1,000 Input Tokens (Coding workload)

Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).

Figures represent median (P50) measurement over the past 72 hours to reflect sustained changes in performance.

Latency vs. Output Speed: Llama 3.3 70B Providers

Latency: seconds to first token received · Output speed: output tokens per second · 1,000 Input Tokens (Coding workload)

Most attractive quadrant

Size represents Price (USD per M Tokens)

Amazon

Azure

Cloudflare

CompactifAI

Databricks

DeepInfra (Turbo, FP8)

Fireworks

FriendliAI

Google Vertex

Groq

Hyperbolic

Lightning AI

Nebius Base

Novita

Parasail (FP8)

SambaNova

Scaleway

Snowflake

Together.ai Turbo

Weights & Biases

Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).

Time to first token received, in seconds, after API request sent. For reasoning models which share reasoning tokens, this will be the first reasoning token. For models which do not support streaming, this represents time to receive the completion.

Price per token, represented as USD per million Tokens. Price is a blend of cache hit, input, and output token prices using the selected ratio.

Figures represent median (P50) measurement over the past 72 hours to reflect sustained changes in performance.

Latency

Measured by Time (seconds) to First Token

Time to First Token: Llama 3.3 70B Providers

Seconds to first token received · Lower is better · 1,000 Input Tokens (Coding workload)

Figures represent median (P50) measurement over the past 72 hours to reflect sustained changes in performance.

End-to-End Response Time

Seconds to output 500 tokens, calculated based on time to first token, 'thinking' time for reasoning models, and output speed

End-to-End Response Time: Llama 3.3 70B Providers

Seconds to output 500 tokens, including reasoning model 'thinking' time · Lower is better · 1,000 Input Tokens (Coding workload)

Input processing time

'Thinking' time (reasoning models)

Outputting time

Seconds to receive a 500 token response. Key components:

Input time: Time to receive the first response token
Thinking time (only for reasoning models): Time reasoning models spend outputting tokens to reason prior to providing an answer. Amount of tokens based on the average reasoning tokens across a diverse set of 60 prompts (methodology details).
Answer time: Time to generate 500 output tokens, based on output speed

For fair comparison, the number of reasoning tokens is standardized across all providers for each model based on the model's representative query token counts.

Figures represent median (P50) measurement over the past 72 hours to reflect sustained changes in performance.

API Features

Function (Tool) Calling & JSON Mode: Llama 3.3 70B Providers

Models	Function calling	JSON Mode
SambaNova
Amazon
Snowflake
Groq
Nebius Base
Fireworks
FriendliAI
Hyperbolic
Google Vertex
Weights & Biases
CompactifAI
Databricks
Together.ai Turbo
Lightning AI
Azure
Scaleway
DeepInfra (Turbo, FP8)
Novita
Cloudflare
Parasail (FP8)

Indicates whether the provider supports function calling in their API. Function calling is also known as 'Tool Calling'.

Indicates whether the provider supports JSON mode in their API. When JSON mode is enabled, the models will always return a valid JSON object.

Context Window: Llama 3.3 70B Providers

Context window: tokens limit · Higher is better

Maximum number of combined input & output tokens. Output tokens commonly have a significantly lower limit (varied by model).

While models have their own context window, in cases this is limited by providers.

Summary Table of Key Comparison Metrics

Reasoning models are indicated by a lightbulb icon

		Features		Price	Speed	Latency	End-to-End Response Time

SambaNova	Llama 3.3 70B	128k	Open	$0.75	435	0.70	1.84	N/A
Amazon Bedrock	Llama 3.3 70B	128k	Open	$0.71	193	0.69	3.28	N/A
Snowflake Snowflake	Llama 3.3 70B Snowflake	8k	Open	$0.58	180	0.52	3.30	N/A
Groq	Llama 3.3 70B	131k	Open	$0.64	494	0.27	1.29	N/A
Nebius Base	Llama 3.3 70B Base	128k	Open	$0.20	48	0.63	11.05	N/A
Fireworks	Llama 3.3 70B	131k	Open	$0.90	178	0.43	3.24	N/A
FriendliAI	Llama 3.3 70B	128k	Open	$0.60	151	0.37	3.67	N/A
Hyperbolic	Llama 3.3 70B	131k	Open	$0.40	104	0.91	5.73	N/A
Google Vertex	Llama 3.3 70B Vertex	128k	Open	$0.72	181	0.23	3.00	N/A
Weights & Biases	Llama 3.3 70B	128k	Open	$0.71	99	0.46	5.50	N/A
CompactifAI	Llama 3.3 70B	128k	Open	$0.40	325	0.28	1.82	N/A
Databricks	Llama 3.3 70B	128k	Open	$0.75	70	0.49	7.60	N/A
Together.ai Turbo	Llama 3.3 70B Turbo	131k	Open	$0.88	84	1.17	7.12	N/A
Lightning AI	Llama 3.3 70B	128k	Open	$0.30	64	0.25	8.12	N/A
Microsoft Azure	Llama 3.3 70B	128k	Open	$0.71	189	0.98	3.62	N/A
Scaleway	Llama 3.3 70B	100k	Open	$1.05	92	0.68	6.10	N/A
DeepInfra (Turbo, FP8)	Llama 3.3 70B (Turbo, FP8)	131k	Open	$0.16	23	0.63	22.18	N/A
Novita	Llama 3.3 70B	131k	Open	$0.20	39	0.65	13.52	N/A
Cloudflare	Llama 3.3 70B	24k	Open	$0.78	39	0.47	13.26	N/A
Parasail (FP8)	Llama 3.3 70B (FP8)	131k	Open	$0.29	34	0.56	15.47	N/A

Frequently Asked Questions

Common questions about Llama 3.3 Instruct 70B providers

Llama 3.3 Instruct 70B is available through 20 API providers: SambaNova, Amazon, Snowflake, Groq, Nebius Base, Fireworks, FriendliAI, Hyperbolic, Google Vertex, Weights & Biases, CompactifAI, Databricks, Together.ai Turbo, Lightning AI, Azure, Scaleway, DeepInfra (Turbo, FP8), Novita, Cloudflare, and Parasail (FP8). Each provider offers different performance characteristics and pricing.

Llama 3.3 Instruct 70B is currently available through 20 API providers that we benchmark and track.

The fastest providers for Llama 3.3 Instruct 70B by output speed are Groq (493.6 t/s), SambaNova (435.4 t/s), and CompactifAI (324.6 t/s). Output speed measures how quickly tokens are generated after the model starts responding.

The providers with the lowest time to first token for Llama 3.3 Instruct 70B are Google Vertex (0.23s), Lightning AI (0.25s), and Groq (0.27s). Lower latency means faster initial response time.

The most affordable providers for Llama 3.3 Instruct 70B by blended price are DeepInfra (Turbo, FP8) ($0.16 per 1M tokens), Nebius Base ($0.20 per 1M tokens), and Novita ($0.20 per 1M tokens). Blended price uses a 3:1 input to output token ratio.

The providers with the lowest input token pricing for Llama 3.3 Instruct 70B are DeepInfra (Turbo, FP8) ($0.10 per 1M input tokens), Nebius Base ($0.13 per 1M input tokens), and Novita ($0.14 per 1M input tokens).

The providers with the lowest output token pricing for Llama 3.3 Instruct 70B are Lightning AI ($0.30 per 1M output tokens), DeepInfra (Turbo, FP8) ($0.32 per 1M output tokens), and Nebius Base ($0.40 per 1M output tokens).

Prices for Llama 3.3 Instruct 70B vary up to 6.8x across providers. The most affordable is DeepInfra (Turbo, FP8) at $0.16 per 1M tokens, while Scaleway charges $1.05 per 1M tokens.

Output speed for Llama 3.3 Instruct 70B varies significantly across providers. Groq is the fastest at 493.6 t/s, which is 21.3x faster than DeepInfra (Turbo, FP8) at 23.2 t/s.

12 of 20 providers support JSON mode for Llama 3.3 Instruct 70B: SambaNova, Groq, Nebius Base, Fireworks, FriendliAI, Google Vertex, Weights & Biases, Together.ai Turbo, Scaleway, DeepInfra (Turbo, FP8), Novita, and Cloudflare.

18 of 20 providers support function calling for Llama 3.3 Instruct 70B: SambaNova, Amazon, Groq, Nebius Base, Fireworks, FriendliAI, Hyperbolic, Google Vertex, Weights & Biases, Databricks, Together.ai Turbo, Lightning AI, Azure, Scaleway, DeepInfra (Turbo, FP8), Novita, Cloudflare, and Parasail (FP8).

The best provider for Llama 3.3 Instruct 70B depends on your priorities: Groq offers the highest output speed, Google Vertex has the lowest latency, and DeepInfra (Turbo, FP8) provides the most competitive pricing.

When choosing a provider for Llama 3.3 Instruct 70B, consider: output speed (for throughput-intensive tasks), latency (for interactive applications requiring quick first responses), pricing (for cost-sensitive workloads), and API features like JSON mode or function calling.

Yes, provider performance can vary over time due to infrastructure changes, load balancing, and updates. We continuously benchmark all providers and display historical performance trends in the "Over Time" charts.

For information about Llama 3.3 Instruct 70B's intelligence, capabilities, modalities, and how it compares to other models, see the model overview page. View model overview

Llama 3.3 Instruct 70B API Provider Benchmarking & Analysis

Fastest

Lowest Latency

Lowest Price

Pricing

Pricing: Cache Hit, Input, and Output

Cache Hit

Input Price

Cache Pricing by Provider

Output Price

Pricing: Blended Price

Price

Cache Hit

Cache Pricing by Provider

Median

Speed vs. Price

Output Speed

Price

Cache Hit

Cache Pricing by Provider

Median

Speed

Output Speed: Llama 3.3 70B Providers

Output Speed

Median

Latency vs. Output Speed: Llama 3.3 70B Providers

Output Speed

Latency (Time to First Token)

Price

Median

Latency

Time to First Token: Llama 3.3 70B Providers

Latency (Time to First Token)

Median

End-to-End Response Time

End-to-End Response Time: Llama 3.3 70B Providers

End-to-End Response Time

Standardized Reasoning Tokens

Median

API Features

Function (Tool) Calling & JSON Mode: Llama 3.3 70B Providers

Function (Tool) Calling

JSON Mode

Context Window: Llama 3.3 70B Providers

Context window

Variance between providers

Summary Table of Key Comparison Metrics

Frequently Asked Questions

Where can I access Llama 3.3 Instruct 70B?

How many API providers offer Llama 3.3 Instruct 70B?

Which provider is fastest for Llama 3.3 Instruct 70B?

Which provider has the lowest latency for Llama 3.3 Instruct 70B?

Which provider is cheapest for Llama 3.3 Instruct 70B?

Which provider has the lowest input price for Llama 3.3 Instruct 70B?

Which provider has the lowest output price for Llama 3.3 Instruct 70B?

How much do prices vary across Llama 3.3 Instruct 70B providers?

How much does speed vary across Llama 3.3 Instruct 70B providers?

Which Llama 3.3 Instruct 70B providers support JSON mode?

Which Llama 3.3 Instruct 70B providers support function calling?

Which is the best provider for Llama 3.3 Instruct 70B?

How do I choose a provider for Llama 3.3 Instruct 70B?

Does provider performance for Llama 3.3 Instruct 70B change over time?

What are the overall capabilities of Llama 3.3 Instruct 70B?