October 14, 2026

Welcome to the era of embodied AI: Introducing AA-Robotics v0.1

We are launching AA-Robotics v0.1, our research preview that measures frontier language model performance on understanding, planning, and acting in the physical world. In v0.1, each model is given control of a 6-axis robot arm (5 joints and a gripper) and a simple objective. We’re releasing results for our first eight tasks today.

AA-Robotics v0.1

Introducing AA-Robotics v0.1

Since the beginning of 2026, frontier language models have made enormous progress in their ability to control robot arms. We now believe that the future of robotics is likely to involve strong generalist agents, and conclude that tracking progress of language models for robotics tasks is going to be important - even if frontier language models are unsuitable for production robotics use on their own.

AA-Robotics v0.1 is our research preview of the simplest experiment we have designed to test this - language models in coding agent harnesses directly controlling a robot arm. Each model is given control of a 6-axis robot arm (5 joints and a gripper) and a simple objective for each task.

We find that frontier language models in coding agent harnesses have indeed recently become capable of directly manipulating robot arms for simple tasks. Today, they are far too slow and expensive for any kind of production usage - nearly two orders of magnitude worse than human baselines. We expect this to change quickly.

GPT-6 Astra places first in AA-Robotics v0.1 with a score of 69, followed by GPT-6.1 Sol, Gemini 4 Argon, Claude Opus 5.5 and Claude Fable 5.1. GPT-6 Astra took 30 minutes and cost $16.36 on average per task.

AA-Robotics Score

Mean score per task (0 to 100) · All tasks

Each attempt is scored out of 100: completing the task scores 100, and a partial result earns points for the progress made.

The eight tasks

AA-Robotics v0.1 initially includes eight tabletop tasks, modeled on everyday activities, to evaluate a range of robotics capabilities: control and dexterity, such as precise grasping and handling objects, spatial reasoning, such as understanding how objects interact with one another, and adaptability, measuring how models respond to the unexpected.

Results

Results per Task

Full score (100)
Partial (1 to 99)
Zero (0)

Task	GPT-6 Astra	GPT-6.1 Sol	Gemini 4 Argon	Claude Opus 5.5	Claude Fable 5.1
Press the red button	Full score, 100 of 100	Full score, 100 of 100	Full score, 100 of 100	Full score, 100 of 100	Full score, 100 of 100
Place the crocodile toy in the rectangle	Full score, 100 of 100	Full score, 100 of 100	Full score, 100 of 100	Full score, 100 of 100	Full score, 100 of 100
Pour the drink into the cup	Full score, 100 of 100	Zero, 0 of 100	Full score, 100 of 100	Partial, 25 of 100	Partial, 30 of 100
Tidy the blocks into the box	Partial, 40 of 100	Full score, 100 of 100	Partial, 40 of 100	Partial, 40 of 100	Partial, 40 of 100
Complete the shape matching challenge	Partial, 44 of 100	Full score, 100 of 100	Zero, 0 of 100	Partial, 38 of 100	Full score, 100 of 100
Write hello world	Full score, 100 of 100	Full score, 100 of 100	Partial, 45 of 100	Partial, 45 of 100	Zero, 0 of 100
Build the tallest tower	Partial, 71 of 100	Zero, 0 of 100	Partial, 57 of 100	Partial, 25 of 100	Zero, 0 of 100
Measure the box	Zero, 0 of 100	Partial, 17 of 100	Zero, 0 of 100	Partial, 17 of 100	Zero, 0 of 100
AA-Robotics Score	69.4	64.6	55.3	48.8	46.3

Each attempt is scored out of 100: completing the task scores 100, and a partial result earns points for the progress made.

GPT models run in Codex, Claude models in Claude Code, and Gemini 4 Argon in Antigravity, all at the high effort setting.

Task

Model

Press the button with the gripper.

Full score in 2:12

0:00 / 0:00Movement only

How AA-Robotics works

Models run in an isolated container with brief instructions: how to reach the robot arm (via a simple HTTP API), which arm it is (a Hiwonder NexArm, with no further specifications), and how to submit an attempt as complete. They can view the scene through four cameras, three mounted around the table and one on the arm’s wrist, and can write and run code in the container to plan and send movements.

Once a model has submitted a task for review, the end state is graded against the task’s pass criteria. A pass scores 100, with points awarded for partial success in the case of an incomplete task. All task attempts are zero-shot.

Time & cost

Today’s language models are roughly 100 times slower than a human on AA-Robotics v0.1, taking about half an hour per task compared with a human average of 20 seconds. In aggregate, four of the five models tested look similar, at about 30 minutes per task, while Gemini 4 Argon is the slowest at 52 minutes. However, these averages hide large differences between tasks.

GPT-6 Astra pressed the red button in 2.2 minutes, whereas Claude Fable 5.1 spent about ten minutes understanding and planning before its first action. The largest difference came on the crocodile placement task, where GPT-6.1 Sol declared itself complete after five minutes, compared with 45 for Claude Opus 5.5, a ninefold difference. Across all tasks, Gemini 4 Argon was the slowest to act, spending an average of 21 minutes observing and reasoning before its first action, nearly twice as long as any other model.

Time per Task

Average time per task (minutes) · All tasks

AA-Robotics Score vs. Time per Task

Average time per task (minutes) · All tasks

Most attractive quadrant

Model Pareto frontier

The gap between humans and models is narrower on cost: the most cost-effective model costs about ten times as much as a person paid the average US hourly wage, against nearly 100 times as long on time. Between models, GPT-6.1 Sol is the cheapest, at $1.93 per task on average, while GPT-6 Astra, the most expensive, costs $16.36, more than eight times as much as Sol, for a score only four points higher (69 against 65). Given the novelty of the arm and scene, and the large volume of visual information the models must process, most of the cost lands on input: 70% of the spend goes on context.

Cost per Task

Average cost per task (USD) · All tasks

AA-Robotics Score vs. Cost per Task

Average cost per task (USD) · All tasks

Most attractive quadrant

Model Pareto frontier

Human cost is the by-hand task time priced at $37.81 per hour, the average hourly earnings of all employees on US private nonfarm payrolls (US Bureau of Labor Statistics, preliminary data for September 2026).

How each model spent its time

As expected, models spend a large portion of the initial task time calibrating the arm and understanding the scene, continuing this up until taking the first action. After this point, the majority of time is spent planning movement, with relatively short, precise arm movements between each longer planning step. Camera views and note writing are relatively small fractions of the overall time.

Interestingly, models often continue to refine their attempts long after reaching a passing state. GPT-6.1 Sol, for example, builds a passing tower after ~17 minutes, only to subsequently fail the task. It remains to be seen how models will improve their ability to reason about task completion in physical space.

Activity Through Each Run

Minutes into the run

Task

0:00 elapsed0 of 5 finished

Understanding the Scene, Arm, and Movement
Planning Movements
Reviewing Cameras
Arm Moving
Other

First Action
First Reached a Passing State
Irrecoverable Mistake

Other covers waiting on running code, writing notes, and the final report.

The first action is the first press, grasp, push, pour, or pen stroke of a run.

An irrecoverable mistake means the model can no longer achieve full marks.

Gemini 4 Argon's activity times are estimated: Antigravity does not timestamp each step, so steps are placed between the arm commands, which are timed exactly, in proportion to their tokens.

Physical accuracy

Grasping and fine manipulation were key skills to test in AA-Robotics v0.1, as they point the way not only to more precise control tasks, but more importantly to tool use and the far wider range of capabilities it unlocks.

GPT-6 Astra and Claude Fable 5.1 performed best with the highest proportion of successful attempts (~45%), indicating a degree of close control, whilst GPT-6.1 Sol, scored worst, at 28%. Gemini 4 Argon fell in between at 36%, with many of its misses closing just above or beside the block it was reaching for.

Outcome of Every Grasp Attempt

Share of grasp attempts (%) · 6 tasks involving grasping

Successful Grasp
Missed, Partial, or Wrong-Object Grasp

GPT-6 Astra

45%

55%

n = 95

Claude Fable 5.1

44%

56%

n = 89

Claude Opus 5.5

40%

60%

n = 100

Gemini 4 Argon

36%

64%

n = 75

GPT-6.1 Sol

28%

72%

n = 111

Grasps were attempted in the Place Crocodile, Pour Drink, Tidy Box, Match Shapes, Build Tower, and Measure Box tasks. Successful is defined as holding the intended object through its lift or carry, and missed, partial, or wrong-object where the gripper did not close on the object, closed on it before it slipped, dropped, or could not be lifted, or took a different object by mistake.

Mistake wall

Hover over a clip to see what went wrong and click a clip to enlarge it. We’ve highlighted a few of our favorite examples!

Misjudging success or failure - and when to stop

Poor physical reasoning

Missed grasps, misplacements and drops

Read the latest

Artificial Analysis Decision Index v1.0

The Artificial Analysis Decision Index benchmarks decision models, a new category of models that answer structured decisions in a single call with a probability for each option. Version 1.0 combines four evaluations across classification, tool calling, knowledge, hallucination, and reasoning.

October 13, 2026

Announcing AA-3D: a benchmark for AI models building interactive 3D experiences

AA-3D is a new benchmark of how well AI models build interactive 3D experiences. Each model turns a written brief into a working three.js scene in an offline sandbox, and our recruited panel compares the scenes in blind pairwise votes.

October 12, 2026

Announcing FrontierCode on Artificial Analysis: can AI models write code maintainers would merge?

Cognition's FrontierCode 1.1 tests whether AI models make code changes that maintainers of real open source repositories would merge. Leading models still fail more than 40% of the time.