Mistral Small 3.1 vs. Llama 3.1 Instruct 405B
Comparison between Mistral Small 3.1 and Llama 3.1 Instruct 405B across intelligence, price, speed, context window and more.
For details relating to our methodology, see our Methodology page.
Highlights
Intelligence
Artificial Analysis Intelligence Index; Higher is better
Speed
Output Tokens per Second; Higher is better
Price
USD per 1M Tokens; Lower is better
Parallel Queries:
Prompt Length:
Model Comparison
| Metric |  Mistral Small 3.1 |  Llama 3.1 Instruct 405B | Analysis |
|---|---|---|---|
| Creator | |||
| Context Window | 128k tokens (~192 A4 pages of size 12 Arial font) | 128k tokens (~192 A4 pages of size 12 Arial font) | Both Mistral Small 3.1 and Llama 3.1 Instruct 405B have the same sized context window |
| Release Date | March, 2025 | July, 2024 | Mistral Small 3.1 has a more recent release date than Llama 3.1 Instruct 405B |
| Parameters | 24B | 405B | Mistral Small 3.1 is smaller than Llama 3.1 Instruct 405B |
| Image Input Support | Yes | No | Mistral Small 3.1 has image input support while Llama 3.1 Instruct 405B does not |
| Open Source (Weights) | Both Mistral Small 3.1 and Llama 3.1 Instruct 405B are open source | ||
| License | |||
| License Supports Commercial Use Without Restrictions | Yes | Yes | Both Mistral Small 3.1 and Llama 3.1 Instruct 405B have license supports commercial use without restrictions |
Intelligence
Artificial Analysis Intelligence Index
+ Add model from specific provider
Intelligence Index incorporates 7 evaluations spanning reasoning, knowledge, math & coding
Artificial Analysis Intelligence Index: Combination metric covering multiple dimensions of intelligence - the simplest way to compare how smart models are. Version 2 was released in Feb '25 and includes: MMLU-Pro, GPQA Diamond, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500. See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Artificial Analysis Intelligence Index by Model Type
+ Add model from specific provider
Intelligence Index incorporates 7 evaluations spanning reasoning, knowledge, math & coding
Non-Reasoning Model
Artificial Analysis Intelligence Index: Combination metric covering multiple dimensions of intelligence - the simplest way to compare how smart models are. Version 2 was released in Feb '25 and includes: MMLU-Pro, GPQA Diamond, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500. See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Artificial Analysis Intelligence Index by Open Weights vs Proprietary
+ Add model from specific provider
Intelligence Index incorporates 7 evaluations spanning reasoning, knowledge, math & coding
Open Weights
Artificial Analysis Intelligence Index: Combination metric covering multiple dimensions of intelligence - the simplest way to compare how smart models are. Version 2 was released in Feb '25 and includes: MMLU-Pro, GPQA Diamond, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500. See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Open Weights: Indicates whether the model weights are available. Models are labelled as 'Commercial Use Restricted' if the weights are available but commercial use is limited (typically requires obtaining a paid license).
Artificial Analysis Coding Index
+ Add model from specific provider
Represents the average of coding benchmarks in the Artificial Analysis Intelligence Index (LiveCodeBench & SciCode)
Artificial Analysis Coding Index: Represents the average of coding evaluations in the Artificial Analysis Intelligence Index. Currently includes: LiveCodeBench, SciCode. See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Artificial Analysis Math Index
+ Add model from specific provider
Represents the average of math benchmarks in the Artificial Analysis Intelligence Index (AIME 2024 & Math-500)
Artificial Analysis Math Index: Represents the average of math evaluations in the Artificial Analysis Intelligence Index. Currently includes: AIME, MATH-500. See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Intelligence Evaluations
+ Add model from specific provider
Intelligence evaluations measured independently by Artificial Analysis; Higher is better
Results claimed by AI Lab (not yet independently verified)
MMLU-Pro (Reasoning & Knowledge)
GPQA Diamond (Scientific Reasoning)
Humanity's Last Exam (Reasoning & Knowledge)
LiveCodeBench (Coding)
SciCode (Coding)
HumanEval (Coding)
MATH-500 (Quantitative Reasoning)
AIME 2024 (Competition Math)
Multilingual Index (Artificial Analysis)
Artificial Analysis Intelligence Index: Combination metric covering multiple dimensions of intelligence - the simplest way to compare how smart models are. Version 2 was released in Feb '25 and includes: MMLU-Pro, GPQA Diamond, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500. See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Intelligence vs. Price
+ Add model from specific provider
Artificial Analysis Intelligence Index (Version 2, released Feb '25); Price: USD per 1M Tokens
Most attractive quadrant
Mistral Small 3.1
Llama 3.1 405B
Artificial Analysis Intelligence Index: Combination metric covering multiple dimensions of intelligence - the simplest way to compare how smart models are. Version 2 was released in Feb '25 and includes: MMLU-Pro, GPQA Diamond, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500. See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Price: Price per token, represented as USD per million Tokens. Price is a blend of Input & Output token prices (3:1 ratio).
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Intelligence vs. Output Speed
+ Add model from specific provider
Artificial Analysis Intelligence Index (Version 2, released Feb '25); Output Speed: Output Tokens per Second
Most attractive quadrant
Mistral Small 3.1
Llama 3.1 405B
Artificial Analysis Intelligence Index: Combination metric covering multiple dimensions of intelligence - the simplest way to compare how smart models are. Version 2 was released in Feb '25 and includes: MMLU-Pro, GPQA Diamond, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500. See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Intelligence vs. Total Response Time
+ Add model from specific provider
Artificial Analysis Intelligence Index (Version 2, released Feb '25); End-to-End Seconds to Output 100 Tokens; Lower is better
Most attractive quadrant
Mistral Small 3.1
Llama 3.1 405B
Artificial Analysis Intelligence Index: Combination metric covering multiple dimensions of intelligence - the simplest way to compare how smart models are. Version 2 was released in Feb '25 and includes: MMLU-Pro, GPQA Diamond, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500. See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Total Response Time: Time to receive a 100 token response. Calculated based on Latency (time to receive first token) and Output Speed (output tokens per second).
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Context Window
Context Window
+ Add model from specific provider
Context Window: Tokens Limit; Higher is better
Context window: Maximum number of combined input & output tokens. Output tokens commonly have a significantly lower limit (varied by model).
Intelligence vs. Context Window
+ Add model from specific provider
Artificial Analysis Intelligence Index (Version 2, released Feb '25); Context Window: Tokens Limit
Most attractive quadrant
Mistral Small 3.1
Llama 3.1 405B
Artificial Analysis Intelligence Index: Combination metric covering multiple dimensions of intelligence - the simplest way to compare how smart models are. Version 2 was released in Feb '25 and includes: MMLU-Pro, GPQA Diamond, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500. See Intelligence Index methodology for further details, including a breakdown of each evaluation and how we run them.
Context window: Maximum number of combined input & output tokens. Output tokens commonly have a significantly lower limit (varied by model).
Pricing
Pricing: Input and Output Prices
+ Add model from specific provider
Price: USD per 1M Tokens
Input price
Output price
Input Price: Price per token included in the request/message sent to the API, represented as USD per million Tokens.
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Pricing: Cached Input Prompts
+ Add model from specific provider
Price: USD per 1M Tokens
Input (standard)
Cache Write
Cache Hit
Cache Storage per Hour
Output (standard)
Input Price: Price per token included in the request/message sent to the API, represented as USD per million Tokens.
Cache Write: One-time cost charged when storing a prompt in the cache for future reuse, represented as USD per million tokens.
Cache Hit: Price per token for cached prompts (previously processed), typically offering a significant discount compared to regular input price, represented as USD per million tokens.
Cache Storage per Hour: Cost to maintain tokens in cache storage, charged per million tokens per hour. Currently only applicable to Google's Gemini models.
Output Price: Price per token generated by the model (received from the API), represented as USD per million Tokens.
Pricing: Image Input Pricing
+ Add model from specific provider
Image Input Price: USD per 1k images at 1MP (1024x1024)
Price per 1k 1MP images: Price for 1,000 images at a resolution of 1 Megapixel (1024 x 1024) processed by the model.
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Performance Summary
Output Speed vs. Price
+ Add model from specific provider
Output Speed: Output Tokens per Second; Price: USD per 1M Tokens
Most attractive quadrant
Mistral Small 3.1
Llama 3.1 405B
Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).
Price: Price per token, represented as USD per million Tokens. Price is a blend of Input & Output token prices (3:1 ratio).
Latency vs. Output Speed
+ Add model from specific provider
Latency: Seconds to First Token Received; Output Speed: Output Tokens per Second
Most attractive quadrant
Mistral Small 3.1
Llama 3.1 405B
Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).
Latency: Time to first token of tokens received, in seconds, after API request sent. For models which do not support streaming, this represents time to receive the completion.
Price: Price per token, represented as USD per million Tokens. Price is a blend of Input & Output token prices (3:1 ratio).
Speed
Measured by Output Speed (tokens per second)
Output Speed
+ Add model from specific provider
Output Tokens per Second; Higher is better
Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Output Speed by Input Token Count (Context Length)
+ Add model from specific provider
Output Tokens per Second; Higher is better
100 input tokens
1k input tokens
10k input tokens
100k input tokens
Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).
Input Tokens Length: Length of tokens provided in the request. See Prompt Options above to see benchmarks of different input prompt lengths across other charts.
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Output Speed Variance
+ Add model from specific provider
Output Tokens per Second; Results by percentile; Higher is better
Median, Other points represent 5th, 25th, 75th, 95th Percentiles respectively
Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).
Boxplot: Shows variance of measurements
Output Speed, Over Time
+ Add model from specific provider
Output Tokens per Second; Higher is better
Output Speed: Tokens per second received while the model is generating tokens (ie. after first chunk has been received from the API for models which support streaming).
Over time measurement: Median measurement per day, based on 8 measurements each day at different times. Labels represent start of week's measurements.
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Latency
Measured by Time (seconds) to First Token
Latency
+ Add model from specific provider
Seconds to First Token Received; Lower is better
Latency: Time to first token of tokens received, in seconds, after API request sent. For models which do not support streaming, this represents time to receive the completion.
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Latency by Input Token Count (Context Length)
+ Add model from specific provider
Seconds to First Token Received; Lower is better
100 input tokens
1k input tokens
10k input tokens
100k input tokens
Input Tokens Length: Length of tokens provided in the request. See Prompt Options above to see benchmarks of different input prompt lengths across other charts.
Latency: Time to first token of tokens received, in seconds, after API request sent. For models which do not support streaming, this represents time to receive the completion.
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Latency Variance
+ Add model from specific provider
Seconds to First Token Received; Results by percentile; Lower is better
Median, Other points represent 5th, 25th, 75th, 95th Percentiles respectively
Latency: Time to first token of tokens received, in seconds, after API request sent. For models which do not support streaming, this represents time to receive the completion.
Boxplot: Shows variance of measurements
Total Response Time
Time to receive 100 tokens output, calculated from latency and output speed metrics
Total Response Time
+ Add model from specific provider
End-to-End Seconds to Output 100 Tokens; Lower is better
Total Response Time: Time to receive a 100 token response. Calculated based on Latency (time to receive first token) and Output Speed (output tokens per second).
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Total Response Time by Input Token Count (Context Length)
+ Add model from specific provider
End-to-End Seconds to Output 100 Tokens; Lower is better
100 input tokens
1k input tokens
10k input tokens
100k input tokens
Input Tokens Length: Length of tokens provided in the request. See Prompt Options above to see benchmarks of different input prompt lengths across other charts.
Total Response Time: Time to receive a 100 token response. Calculated based on Latency (time to receive first token) and Output Speed (output tokens per second).
Figures represent performance of the model's first-party API (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Meta's Llama models).
Total Response Time Variance
+ Add model from specific provider
Total Response Time: End-to-End Seconds to Output 100 Tokens; Results by percentile; Lower is better
Median, Other points represent 5th, 25th, 75th, 95th Percentiles respectively
Total Response Time: Time to receive a 100 token response. Calculated based on Latency (time to receive first token) and Output Speed (output tokens per second).
Boxplot: Shows variance of measurements
Comparisons to Mistral Small 3.1
o1- GPT-4o (Nov '24)
- GPT-4o mini
- o3-mini (high)
- o1-pro
- Llama 3.3 Instruct 70B
- Llama 3.1 Instruct 8B
Gemini 2.0 Flash (Feb '25)- Gemma 3 27B Instruct
- Gemini 2.5 Pro Experimental (Mar' 25)
Claude 3.7 Sonnet (Standard)- Claude 3.7 Sonnet (Extended Thinking)
- Mistral Large 2 (Nov '24)
DeepSeek R1- DeepSeek V3 0324 (Mar' 25)
Grok 3 Reasoning Beta- Grok 3
Nova Pro- Nova Micro
Command A
QwQ 32B- DeepSeek V3
Comparisons to Llama 3.1 405B
- o1
- GPT-4o (Nov '24)
- GPT-4o mini
- o3-mini (high)
- o1-pro
- Llama 3.3 Instruct 70B
- Llama 3.1 Instruct 8B
- Gemini 2.0 Flash (Feb '25)
- Gemma 3 27B Instruct
- Gemini 2.5 Pro Experimental (Mar' 25)
- Claude 3.7 Sonnet (Standard)
- Claude 3.7 Sonnet (Extended Thinking)
- Mistral Large 2 (Nov '24)
- Mistral Small 3.1
- DeepSeek R1
- DeepSeek V3 0324 (Mar' 25)
- Grok 3 Reasoning Beta
- Grok 3
- Nova Pro
- Nova Micro
- Command A
- QwQ 32B
- DeepSeek V3
Further details
Sonar Reasoning Pro
Phi-4
LFM 40B
Solar Mini
DBRX Instruct
MiniMax-Text-01
Llama 3.1 Nemotron Instruct 70B
Llama 3.1 Tulu3 405B
Reka Flash (Sep '24)
Jamba 1.5 Large
Arctic Instruct
Yi-Large
OpenChat 3.5 (1210)Models compared: OpenAI: GPT 4o Audio, GPT 4o Realtime, GPT 4o Speech Pipeline, GPT-3.5 Turbo, GPT-3.5 Turbo (0125), GPT-3.5 Turbo (0314), GPT-3.5 Turbo (1106), GPT-3.5 Turbo Instruct, GPT-4, GPT-4 Turbo, GPT-4 Turbo (0125), GPT-4 Turbo (1106), GPT-4 Vision, GPT-4.5 (Preview), GPT-4o (Aug '24), GPT-4o (ChatGPT), GPT-4o (May '24), GPT-4o (Nov '24), GPT-4o Realtime (Dec '24), GPT-4o mini, GPT-4o mini Realtime (Dec '24), o1, o1-mini, o1-preview, o1-pro, o3, o3-mini, and o3-mini (high), Meta: Code Llama 70B, Llama 2 Chat 13B, Llama 2 Chat 70B, Llama 2 Chat 7B, Llama 3 70B, Llama 3 8B, Llama 3.1 405B, Llama 3.1 70B, Llama 3.1 8B, Llama 3.2 11B (Vision), Llama 3.2 1B, Llama 3.2 3B, Llama 3.2 90B (Vision), and Llama 3.3 70B, Google: Gemini 1.0 Pro, Gemini 1.5 Flash (May), Gemini 1.5 Flash (Sep), Gemini 1.5 Flash-8B, Gemini 1.5 Pro (May), Gemini 1.5 Pro (Sep), Gemini 2.0 Flash, Gemini 2.0 Flash (exp), Gemini 2.0 Flash Thinking exp. (Dec '24), Gemini 2.0 Flash Thinking exp. (Jan '25), Gemini 2.0 Flash-Lite (Feb '25), Gemini 2.0 Flash-Lite (Preview), Gemini 2.0 Pro Experimental, Gemini 2.5 Pro Experimental, Gemini Experimental (Nov), Gemma 2 27B, Gemma 2 9B, Gemma 3 12B, Gemma 3 1B, Gemma 3 27B, Gemma 3 4B, and Gemma 7B, Anthropic: Claude 2.0, Claude 2.1, Claude 3 Haiku, Claude 3 Opus, Claude 3 Sonnet, Claude 3.5 Haiku, Claude 3.5 Sonnet (June), Claude 3.5 Sonnet (Oct), Claude 3.7 Sonnet Thinking, Claude 3.7 Sonnet, and Claude Instant, Mistral: Codestral (Jan '25), Codestral (May '24), Codestral-Mamba, Ministral 3B, Ministral 8B, Mistral 7B, Mistral Large (Feb '24), Mistral Large 2 (Jul '24), Mistral Large 2 (Nov '24), Mistral Medium, Mistral NeMo, Mistral Saba, Mistral Small (Feb '24), Mistral Small (Sep '24), Mistral Small 3, Mistral Small 3.1, Mixtral 8x22B, Mixtral 8x7B, Pixtral 12B, and Pixtral Large, DeepSeek: DeepSeek Coder V2 Lite, DeepSeek LLM 67B (V1), DeepSeek R1, DeepSeek R1 (FP4), DeepSeek R1 Distill Llama 70B, DeepSeek R1 Distill Llama 8B, DeepSeek R1 Distill Qwen 1.5B, DeepSeek R1 Distill Qwen 14B, DeepSeek R1 Distill Qwen 32B, DeepSeek V3, DeepSeek V3 (Mar' 25), DeepSeek-Coder-V2, DeepSeek-V2, DeepSeek-V2.5, DeepSeek-V2.5 (Dec '24), DeepSeek-VL2, and Janus Pro 7B, Perplexity: PPLX-70B Online, PPLX-7B-Online, R1 1776, Sonar, Sonar 3.1 Huge, Sonar 3.1 Large, Sonar 3.1 Small , Sonar Large, Sonar Pro, Sonar Reasoning, Sonar Reasoning Pro, and Sonar Small, xAI: Grok 2, Grok 3, Grok 3 Reasoning Beta, Grok 3 mini, Grok 3 mini Reasoning, Grok Beta, and Grok-1, OpenChat: OpenChat 3.5, Amazon: Nova Lite, Nova Micro, and Nova Pro, Microsoft Azure: Phi-3 Medium 14B, Phi-3 Mini, Phi-4, Phi-4 Mini, and Phi-4 Multimodal, Liquid AI: LFM 1.3B, LFM 3B, and LFM 40B, Upstage: Solar Mini, Solar Pro, and Solar Pro (Nov '24), Databricks: DBRX, MiniMax: MiniMax-Text-01, NVIDIA: Cosmos Nemotron 34B, Llama 3.1 Nemotron 70B, Llama 3.3 Nemotron Nano 8B v1, Llama 3.3 Nemotron Nano 8B v1 (Reasoning), Llama 3.3 Nemotron Super 49B v1, and Llama 3.3 Nemotron Super 49B v1 (Reasoning), IBM: Granite 3.0 2B, OpenVoice: Granite 3.0 8B, Inceptionlabs: Mercury Coder Mini, Mercury Coder Small, and Mercury Instruct, Reka AI: Reka Core, Reka Edge, Reka Flash (Feb '24), Reka Flash, and Reka Flash 3, Other: LLaVA-v1.5-7B, Cohere: Aya Expanse 32B, Aya Expanse 8B, Command, Command A, Command Light, Command R7B, Command-R, Command-R (Mar '24), Command-R+ (Apr '24), and Command-R+, AI21 Labs: Jamba 1.5 Large, Jamba 1.5 Large (Feb '25), Jamba 1.5 Mini, Jamba 1.5 Mini (Feb 2025), Jamba 1.6 Large, Jamba 1.6 Mini, and Jamba Instruct, Snowflake: Arctic, Alibaba: QwQ-32B, QwQ 32B-Preview, Qwen Chat 72B, Qwen Plus, Qwen Turbo, Qwen1.5 Chat 110B, Qwen1.5 Chat 14B, Qwen1.5 Chat 32B, Qwen1.5 Chat 72B, Qwen1.5 Chat 7B, Qwen2 72B, Qwen2 Instruct 7B, Qwen2 Instruct A14B 57B, Qwen2-VL 72B, Qwen2.5 Coder 32B, Qwen2.5 Coder 7B , Qwen2.5 Instruct 14B, Qwen2.5 Instruct 32B, Qwen2.5 72B, Qwen2.5 Instruct 7B, Qwen2.5 Max, Qwen2.5 Max 01-29, Qwen2.5 VL 72B, and Qwen2.5 VL 7B, and 01.AI: Yi-Large.