Artificial Analysis Capability Indexes Methodology
Overview
Artificial Analysis Capability Indexes measure how well models perform on specific professional use cases, such as legal, healthcare, or finance work.
We run every component benchmark independently before combining its scores into the index. Like all evaluation metrics, capability indexes have limitations and may not apply directly to every use case, but they provide a useful synthesis for comparing models on the work that matters to a given domain.
The underlying benchmarks, including how each one is run and scored, are documented in the Intelligence Benchmarking methodology.
Index Breakdowns
Each index targets a single profession or field, such as Legal, Healthcare & Medical, or Finance & Accounting, and weights a set of capabilities by how often each appears in real-world tasks for that field.
Our task weightings draw on an O*NET-style taxonomy of work activities. The table below shows the components and weights for each index.
| Index | Weight | Capability | Evaluations | Description |
|---|---|---|---|---|
| Finance & Accounting Index | 30% | Agentic Knowledge Work | AA-Briefcase v1.1 | Tool use, planning, and multi-step task execution, such as building spreadsheets, running an analysis, or coordinating a workflow |
| 25% | Business Knowledge | AA-Omniscience v1.1 | Domain recall in accounting, corporate finance, economics, and investments | |
| 15% | Reasoning | HLE-Diamond | Multi-step quantitative and analytic reasoning, used for sensitivity analysis, valuation, and structured problem solving | |
| 15% | Non-Hallucination | AA-Omniscience v1.1 | Avoiding fabricated figures or citations | |
| 10% | Agentic Tool Use | AutomationBench-AA | Completing finance workflows across business apps such as spreadsheets, email, and accounting tools without breaking guardrails | |
| 5% | Long-Context | GDP.pdf | Reading and reasoning across long financial filings, deal documents, and PDF reports | |
| Strategy & Ops Index | 35% | Agentic Tool Use | AutomationBench-AA | Completing operations, HR, marketing, sales, and support workflows across business apps without breaking guardrails |
| 30% | Agentic Knowledge Work | AA-Briefcase v1.1 | Tool use, planning, and orchestrating multi-step office workflows end-to-end | |
| 25% | Business Knowledge | AA-Omniscience v1.1 | Working knowledge of business processes, accounting basics, and operational concepts | |
| 5% | Long-Context | GDP.pdf | Holding context across long threads, policies, records, and PDF documents | |
| 5% | Non-Hallucination | AA-Omniscience v1.1 | Avoiding fabricated figures, policies, or records | |
| Legal Index | 30% | Legal Knowledge | AA-Omniscience v1.1 | Recall of statutes, doctrines, and procedure across jurisdictions |
| 25% | Agentic Knowledge Work | AA-Briefcase v1.1 | Running matter-management workflows, drafting pipelines, and tool-augmented research | |
| 15% | Reasoning | HLE-Diamond | Multi-step argumentation, statutory interpretation, and weighing conflicting authorities | |
| 15% | Non-Hallucination | AA-Omniscience v1.1 | Avoiding fabricated case cites or invented statutes | |
| 10% | Long-Context | GDP.pdf | Reading and synthesizing across contracts, discovery productions, case-law packets, and PDF filings | |
| 5% | Agentic Tool Use | AutomationBench-AA | Completing client support and operations workflows across business apps without breaking guardrails | |
| Healthcare & Medical Index | 25% | Medical & Health Knowledge | AA-Omniscience v1.1 | Clinical knowledge across diagnosis, pharmacology, and care pathways |
| 25% | Agentic Knowledge Work | AA-Briefcase v1.1 | Tool use, planning, and orchestrating EHR/pharmacy workflows end-to-end | |
| 15% | Long-Context Reasoning | MLCR-AA | Synthesising findings across long, fragmented patient records and claims files | |
| 15% | Agentic Tool Use | AutomationBench-AA | Completing patient support and operations workflows across business apps without breaking guardrails | |
| 10% | Non-Hallucination | AA-Omniscience v1.1 | Avoiding fabricated drug interactions, doses, or guidelines | |
| 10% | Reasoning | HLE-Diamond | Multi-step clinical reasoning across biology and medicine | |
| Engineering Index | 30% | Engineering Knowledge | AA-Omniscience v1.1 | Domain recall across civil, electrical, mechanical, and other engineering disciplines |
| 20% | Science | CritPt v2 (Hard Subset), Terminal-Bench-Science 0.1 | Research-level physics and expert research workflows across the life, earth, and physical sciences | |
| 20% | Reasoning | HLE-Diamond | Multi-step quantitative reasoning for derivations, sizing calculations, and design trade-offs | |
| 15% | Agentic Coding | Terminal-Bench 4.0, FrontierCode 2.0 | Building, debugging, and maintaining software in real codebases and terminal environments | |
| 15% | Agentic Knowledge Work | AA-Briefcase v1.1 | Tool use, planning, and multi-step execution of engineering deliverables | |
| Economics Index | 35% | Agentic Knowledge Work | AA-Briefcase v1.1 | Tool use, planning, and multi-step execution of analytical deliverables |
| 30% | Economics Knowledge | AA-Omniscience v1.1 | Recall across micro and macroeconomics, public finance, and markets | |
| 25% | Reasoning | HLE-Diamond | Multi-step quantitative and analytic reasoning for modeling, estimation, and inference | |
| 10% | Long-Context Reasoning | GDP.pdf | Reading and reasoning across long PDF reports and research papers |