Artificial Analysis AI Trends

We calculate the Artificial Analysis Progress Index v1 using item response theory, which estimates model capability and task difficulty together from our benchmark results. This lets us compare models tested on different benchmarks over time.

We estimate task difficulty from which models succeed or fail, while estimating each model's capability from its results across tasks.

For example, an older model might have been tested on easier benchmarks and a newer model on harder ones. Models tested on both sets help us estimate the difference in difficulty and compare the older and newer models on the same scale.

Each lab’s line shows its highest-scoring model available at that point in time, using each model’s highest-scoring effort variant. Read the Progress Index v1 methodology.

We calculate the Artificial Analysis Progress Index v1 using item response theory, which estimates model capability and task difficulty together from our benchmark results. This lets us compare models tested on different benchmarks over time.

We estimate task difficulty from which models succeed or fail, while estimating each model's capability from its results across tasks.

For example, an older model might have been tested on easier benchmarks and a newer model on harder ones. Models tested on both sets help us estimate the difference in difficulty and compare the older and newer models on the same scale.

We show the highest-scoring effort variant for each model. Read the Progress Index v1 methodology.

We calculate the Artificial Analysis Progress Index v1 using item response theory, which estimates model capability and task difficulty together from our benchmark results. This lets us compare models tested on different benchmarks over time.

We estimate task difficulty from which models succeed or fail, while estimating each model's capability from its results across tasks.

For example, an older model might have been tested on easier benchmarks and a newer model on harder ones. Models tested on both sets help us estimate the difference in difficulty and compare the older and newer models on the same scale.

Each line shows the lowest cost per Intelligence Index task reached over time among all effort variants that meet or exceed its Progress Index v1 threshold. Read the Progress Index v1 methodology.

We calculate the Artificial Analysis Progress Index v1 using item response theory, which estimates model capability and task difficulty together from our benchmark results. This lets us compare models tested on different benchmarks over time.

We estimate task difficulty from which models succeed or fail, while estimating each model's capability from its results across tasks.

For example, an older model might have been tested on easier benchmarks and a newer model on harder ones. Models tested on both sets help us estimate the difference in difficulty and compare the older and newer models on the same scale.

Each line shows the lowest output price reached over time among all effort variants that meet or exceed its Progress Index v1 threshold. Read the Progress Index v1 methodology.

We calculate the Artificial Analysis Progress Index v1 using item response theory, which estimates model capability and task difficulty together from our benchmark results. This lets us compare models tested on different benchmarks over time.

We estimate task difficulty from which models succeed or fail, while estimating each model's capability from its results across tasks.

For example, an older model might have been tested on easier benchmarks and a newer model on harder ones. Models tested on both sets help us estimate the difference in difficulty and compare the older and newer models on the same scale.

Each line shows the highest output speed reached over time among all effort variants that meet or exceed its Progress Index v1 threshold. Read the Progress Index v1 methodology.

We calculate the Artificial Analysis Progress Index v1 using item response theory, which estimates model capability and task difficulty together from our benchmark results. This lets us compare models tested on different benchmarks over time.

We estimate task difficulty from which models succeed or fail, while estimating each model's capability from its results across tasks.

For example, an older model might have been tested on easier benchmarks and a newer model on harder ones. Models tested on both sets help us estimate the difference in difficulty and compare the older and newer models on the same scale.

Each country’s line shows its highest-scoring model available at that point in time, using each model’s highest-scoring effort variant. Read the Progress Index v1 methodology.

We calculate the Artificial Analysis Progress Index v1 using item response theory, which estimates model capability and task difficulty together from our benchmark results. This lets us compare models tested on different benchmarks over time.

We estimate task difficulty from which models succeed or fail, while estimating each model's capability from its results across tasks.

For example, an older model might have been tested on easier benchmarks and a newer model on harder ones. Models tested on both sets help us estimate the difference in difficulty and compare the older and newer models on the same scale.

Each country’s line shows its highest-scoring open weights model available at that point in time, using each model’s highest-scoring effort variant. Read the Progress Index v1 methodology.

We calculate the Artificial Analysis Progress Index v1 using item response theory, which estimates model capability and task difficulty together from our benchmark results. This lets us compare models tested on different benchmarks over time.

We estimate task difficulty from which models succeed or fail, while estimating each model's capability from its results across tasks.

For example, an older model might have been tested on easier benchmarks and a newer model on harder ones. Models tested on both sets help us estimate the difference in difficulty and compare the older and newer models on the same scale.

Each line shows the highest-scoring model available at that point in time within its license category, using each model’s highest-scoring effort variant. Read the Progress Index v1 methodology.

We calculate the Artificial Analysis Progress Index v1 using item response theory, which estimates model capability and task difficulty together from our benchmark results. This lets us compare models tested on different benchmarks over time.

We estimate task difficulty from which models succeed or fail, while estimating each model's capability from its results across tasks.

For example, an older model might have been tested on easier benchmarks and a newer model on harder ones. Models tested on both sets help us estimate the difference in difficulty and compare the older and newer models on the same scale.

We show the highest-scoring effort variant for each model. Read the Progress Index v1 methodology.