Coding Agent Benchmarks

Codex vs. Mistral Vibe

Comparison between Codex and Mistral Vibe across the Artificial Analysis Coding Agent Index, including benchmark scores, cost, execution time, and token usage.

For details relating to our methodology, see our methodology page.

Explore other comparisons
vs
Artificial Analysis Coding Agent Index v2.1 · Higher is better
Average agent wall time per task · Lower is better
Average API cost per task (USD) · Lower is better

Comparison

Side-by-side comparison of Codex and Mistral Vibe.

Metric
GPT-6.2 Astra (Max)
Mistral Large 4 (High)
Analysis
Agent Harness
Codex
Mistral Vibe
Representative Model
GPT-6.2 Astra (Max)
Mistral Large 4 (High)
Coding Agent Index
53
41
Codex has a higher Coding Agent Index than Mistral Vibe
FrontierCode-AA 2.0
42
20
Codex has a higher FrontierCode-AA 2.0 score than Mistral Vibe
Terminal-Bench 4.1
64
42
Codex has a higher Terminal-Bench 4.1 score than Mistral Vibe
SWE-Atlas-QnA-AA
55
59
Mistral Vibe has a higher SWE-Atlas-QnA-AA score than Codex
Cost per Task
$5.12
$1.34
Mistral Vibe has a lower cost per task than Codex
Time per Task
25.1m
49.0m
Codex has a lower time per task than Mistral Vibe
Turns per Task
51
107.3
Codex has a lower turns per task than Mistral Vibe
Token Usage per Task
4.67M
9.42M
Codex has a lower token usage per task than Mistral Vibe
Cache Hit Rate
95%
92%
Codex has a higher cache hit rate than Mistral Vibe

Models

Evaluated models for Codex and Mistral Vibe.

GPT-6.2 Astra (Max)
max
53
42
64
55
$5.12
25.1m
4.67M
GPT-6.2 Astra (Xhigh)
xhigh
53
43
62
54
$3.20
16.5m
3.09M
GPT-6 Astra (Max)
max
53
43
58
59
$7.63
26.0m
3.47M
GPT-6.2 Astra (High)
high
52
42
60
55
$2.37
15.8m
2.26M
GPT-6.1 Sol (Max)
max
52
42
54
60
$1.51
21.9m
4.33M
GPT-6.2 Sol (Xhigh)
xhigh
50
40
58
51
$0.90
12.6m
3.26M
GPT-6.2 Astra (Medium)
medium
49
40
54
54
$1.20
11.6m
1.02M
GPT-6.1 Terra (Max)
max
49
41
51
54
$1.53
28.1m
14.4M
GPT-6.2 Sol (Max)
max
49
42
58
46
$1.22
17.5m
4.25M
GPT-6.2 Sol (High)
high
47
38
54
51
$0.76
10.6m
2.96M
GPT-6.2 Sol (Medium)
medium
45
36
49
50
$0.57
8.4m
2.28M
GPT-6.2 Sol (Low)
low
37
28
41
42
$0.33
7.2m
1.31M
GPT-6.2 Astra (Low)
low
36
27
44
36
$0.70
7.8m
551k
GPT-6 Luna (Max)
max
26
24
15
38
$0.19
23.5m
10.1M
Mistral Large 4 (High)
high
41
20
42
59
$1.34
49.0m
9.42M
Mistral Large 4 (Medium)
medium
30
18
26
45
$1.02
43.1m
7.73M

Performance

Performance across the Artificial Analysis Coding Agent Index.

Artificial Analysis Coding Agent Index

Artificial Analysis Coding Agent Index v2.1 incorporates 3 benchmarks: FrontierCode-AA 2.0, Terminal-Bench 4.1, and SWE-Atlas-QnA-AA · Higher is better
Color by

Artificial Analysis Coding Agent Index vs. Cost per Task

Artificial Analysis Coding Agent Index vs. average pay-per-token API cost per task (USD)
Color by
Most attractive quadrant
Pareto line

Token Usage

Token consumption across the Artificial Analysis Coding Agent Index.

Token Usage per Task

Average input, cache, and output tokens per task
Prompt cache hit rates can vary significantly by provider routing, which can materially change effective cost.

Artificial Analysis Coding Agent Index vs. Total Tokens

Artificial Analysis Coding Agent Index vs. average total tokens per task
Color by
Most attractive quadrant

Cost

Pay-per-token API cost across the Artificial Analysis Coding Agent Index, based on current per-token pricing.

Cost per Task

Average pay-per-token API cost per task (USD) · Lower is better
Color by

Execution Time

Active agent runtime across the Artificial Analysis Coding Agent Index.

Time per Task

Average agent wall time per task · Lower is better
Color by

Artificial Analysis Coding Agent Index vs. Execution Time

Artificial Analysis Coding Agent Index vs. average agent wall time per task
Color by
Most attractive quadrant