Artificial Analysis Progress Index v1

The Artificial Analysis Progress Index v1 is a single capability scale for language models since 2022. It uses item response theory, a statistical method used to score standardized tests, to estimate one general capability score for each model from our evaluation results, including evaluations that were retired or never part of the Intelligence Index.

The Intelligence Index uses the evaluations most relevant to developers today, so its mix changes as evaluations are added and retired, and scores from different versions can't be compared. Use the Intelligence Index to choose a model today, and the Progress Index v1 to see how capability has changed over time.

The Progress Index v1 builds on prior work placing models on one capability scale, in particular Epoch AI's Epoch Capabilities Index and the capability scale Anthropic published in its Claude Mythos Preview system card.

Methodology

Frontier by Lab

Artificial Analysis Progress Index v1, Over Time

Item response theory estimate of general capability across Artificial Analysis evaluations · Highest-scoring effort variant shown

Nov 2022Oct 2026

We calculate the Artificial Analysis Progress Index v1 using item response theory, which estimates model capability and task difficulty together from our benchmark results. This lets us compare models tested on different benchmarks over time.

We estimate task difficulty from which models succeed or fail, while estimating each model's capability from its results across tasks.

For example, an older model might have been tested on easier benchmarks and a newer model on harder ones. Models tested on both sets help us estimate the difference in difficulty and compare the older and newer models on the same scale.

Each lab’s line shows its highest-scoring model available at that point in time, using each model’s highest-scoring effort variant. Read the Progress Index v1 methodology.

Artificial Analysis Progress Index v1

Frontier by Lab

Artificial Analysis Progress Index v1, Over Time

How the Progress Index v1 is calculated