Replicate: Models Intelligence, Performance & Price
This analysis is intended to support you in choosing the best model provided by Replicate for your use-case.
Most Intelligent
Intelligence Index
Total 2 models
Lowest Time per Task
Time per Intelligence Index task (minutes)
Total 2 models
Lowest Price
Llama 3 8BLlama 3 70BBlended price (per 1M tokens)
Total 2 models
Replicate offers 2 models, each with different intelligence, performance, and pricing characteristics. Below is a comparison of the key metrics across models.
- For latency, Llama 3 8B (0.50s) and Llama 3 70B (0.55s) offer the lowest time to first answer token.
- For pricing, Llama 3 8B ($0.07) and Llama 3 70B ($0.86) offer the lowest blended prices per 1M tokens.
- For context window size, Llama 3 8B (8k) and Llama 3 70B (8k) support the largest context windows on Replicate.
Intelligence
Artificial Analysis Intelligence Index
Evaluations
Agentic knowledge work, Elo
Knowledge
1 - hallucination rate
Agentic SaaS workflows
Agentic scientific research workflows in a terminal
Agentic software engineering in open-source repositories
Reasoning & knowledge
Professional document reasoning, All-pass
Agentic coding & terminal use
Physics reasoning
Quantitative analysis on spreadsheets & documents
Legal agentic work, Hallucination-Gated All-Pass Rate
Agentic business operations
Agentic real-world work tasks, Elo
Agentic interactive 3D experiences, Elo
Product and brand videos designed in code, Elo
Kubernetes incident root-cause analysis
Medical long context reasoning
Visual reasoning
Rich, varied, well-crafted language
Well-structured, scannable, answer-first
Truthful, acknowledges uncertainty, non-sycophantic
Use of filler and generic language
Intelligence Index vs. Price
Pricing
Cost per Intelligence Index Task
Intelligence Index vs. Price
Speed
Average of the single-request and multi-turn agentic workloads measured
Output Speed
Quality
Context Window
Latency
Average of the single-request and multi-turn agentic workloads measured
Time to First Answer Token
Reliability
Share of requests the endpoint failed, across all the single-request and multi-turn agentic workloads measured
Endpoint Error Rate
Key definitions
Frequently Asked Questions
Common questions about Replicate
Replicate offers 2 models that we track: Llama 3 8B and Llama 3 70B.
The model with the lowest time to first answer token on Replicate is Llama 3 8B at 0.50s. Lower latency means faster initial response time.
The most affordable model on Replicate by blended price is Llama 3 8B at $0.07 per 1M tokens (7:2:1 cache hit/input/output ratio).
Prices on Replicate vary up to 12x across models, from $0.07 per 1M tokens for Llama 3 8B to $0.86 per 1M tokens for Llama 3 70B.
Yes, Replicate offers an OpenAI-compatible API, making it easy to switch from OpenAI or use existing OpenAI SDK integrations.
Yes, all 2 models on Replicate are open weight models.
Yes, provider performance can vary over time due to infrastructure changes, load balancing, and updates. We continuously benchmark all providers and display historical performance trends in the "Over Time" charts.
When choosing a model on Replicate, consider: intelligence (for quality-sensitive tasks), time per task (how quickly the model completes an Intelligence Index task), latency (for interactive applications requiring quick first responses), pricing (for cost-sensitive workloads), and features like context window size, JSON mode, or function calling support.