All evaluations

AA-PersonalityBench: AI Personality & Communication Style Benchmark

Private DatasetWritingPersonality
Measures how models communicate rather than whether they are correct: 30 judged personality traits rolled up into 8 indexes, plus 21 measured communication characteristics.
See example tasks

Personality Indexes

Each index takes the mean of its traits’ Elo z-scores, then percentile ranks it from 1 to 100 across all models evaluated in the benchmark.

Personality Indexes

Each axis is one of 8 personality indexes: the mean of its traits’ Elo z-scores, percentile ranked from 1 to 100 across all models evaluated in the benchmark

How Traits Form Personality Indexes

30 questions per personality trait (900 questions total), judged by a panel of models. Each index z-scores its traits’ Elo ratings, inverts traits where needed, calculates the mean, and percentile ranks the result from 1 to 100 across all models evaluated in the benchmark.

Expressiveness
150 questions
Rich, varied, well-crafted language
Lexically VariedEloquentLiterateNatural SoundingMetaphorical
Organization
120 questions
Well-structured, scannable, answer-first
StructuredScannableBottom-Line SummaryAnswer First
Informativeness
150 questions
Practical depth, evidence, actionable content
Information DenseEvidence-BasedResource-SpecificActionable GuidancePragmatic
Transparency
150 questions
Truthful, acknowledges uncertainty, non-sycophantic
TruthfulAdmits UncertaintyDisclaims AssumptionsFlags NuancesSycophantic
Warmth
120 questions
Friendly, humorous, sincere, emotionally attuned
FriendlyHumorousSincereEmotionally Intelligent
Conviction
120 questions
Direct, confident positions and constructive pushback
DirectConfidentOpinionatedConstructive Pushback
Imagination
60 questions
Creative, exploratory, open-ended thinking
CreativeOpen-Ended
Slop
30 questions
Use of filler and generic language
Slop
View all indexes

Example Questions, Responses, and Grading

Explore the 5 public AA-PersonalityBench questions, with answers from the highlighted models. Each answer is judged pairwise against other models’ answers to the same question, on that question’s trait. The grading shown is for one question; trait scores use all 30 questions for each trait.

What's the weirdest social norm we all just accept?

Model A

Great question — here are some strong contenders:

  • Handshakes — We judge a person's character by how they squeeze our hand. If you squeeze too soft = weak. Too hard = aggressive. Wrong angle = weird. It's essentially a secret society ritual we all perform without thinking.

  • Shoes indoors = dirty, but carrying phones everywhere including the bathroom = fine — and we put the phone on our face afterward.

  • Grinding noise of alarm clocks — We chose the most jarring sound possible to transition someone out of unconsciousness, then expect them to be pleasant.

  • "How are you?" / "Good, you?" / "Good." — A mandatory exchange of information neither person wants or believes.

  • Clothing gender coding — A skirt is "masculine" on a Scottish man but "feminine" on anyone else. The fabric didn't change.

  • Blowing out candles on a cake — Everyone watches you spit on a communal dessert, then everyone eats it. We could just... light a small candle on the side.

  • Funerals needing suits — Mourning someone's death requires specific fabric choices. This one seems especially arbitrary.

  • Stripping the (already abstracted) connection to where food comes from — We find raw meat touching other food horrifying at the grocery store, but steak tartare at a restaurant is sophisticated.

The one I find genuinely the weirdest might be the handshake — it's a brief, specific physical ritual we use to assess trustworthiness, and its origins are murky even to historians. We've all just... agreed to this.

What's yours?

Model B

My pick is how we behave when we're physically close to strangers.

On a packed train you can be pressed against someone closely enough to smell their shampoo, and the rule is that you both pretend the other person doesn't exist. Elevators are even stranger. Everyone files in, turns to face the door, and stares at the floor numbers like they're the season finale of something.

The sociologist Erving Goffman called this "civil inattention": a brief glance to acknowledge someone's there, then eyes away to signal "I'm not a threat, and I'm not interested in you." It's actually a clever solution to an awkward situation. But zoom out and it's bizarre: a dozen humans in a metal box, silently agreeing to act like furniture.

It's also surprisingly arbitrary. In a 1962 Candid Camera stunt, people stepped into elevators where actors were all facing the back wall, and they quietly turned around to match them.

Some runners-up:

  • Birthday candles: We have someone blow forcefully across a cake everyone's about to eat, then we applaud. (Someone studied it. The frosting does not come out ahead.)
  • "How are you?": It's a direct question where answering honestly counts as breaking the rules.
  • Lawns: We grow a plant we can't eat, aren't allowed to let grow, and must cut every week. It's a leftover status symbol from European aristocrats flaunting land they didn't need to farm.
  • Salary secrecy: We treat pay as taboo, a norm that mostly benefits whoever signs the paychecks, even though in the US most employees are legally protected when they discuss it.
  • The tooth fairy: We tell kids a stranger sneaks into their bedroom at night to buy their body parts.

What's yours?

Grading summary

MiMo-V2.6-Pro
Question win rate45%23 more · 0 ties · 28 less, of 51 comparisons · ties count half
Humorous percentile64thacross all 30 questions and models
Example verdict
1 of 1 judgeMore Humorous
MiMo-V2.6-Pro vs. Grok 4.7 (Xhigh)

Answer A has more playful, witty moments: the 'secret society ritual' handshake, the alarm clock bit, the 'spit on a communal dessert' candle joke, and the 'We've all just... agreed to this' aside. Its tone is more comedic overall, though a few items are clumsy or awkward. Answer B has some dry wit, like 'choking hazard' and 'tiny, automatic lie', but it's mostly measured and analytical, so it is less overtly humorous.

Answers were blinded: Answer A was MiMo-V2.6-Pro and Answer B was Grok 4.7 (Xhigh).
Claude Opus 5.5 (Max, Default Fallback)
Question win rate35%12 more · 0 ties · 22 less, of 34 comparisons · ties count half
Humorous percentile88thacross all 30 questions and models
Example verdict
2 of 2 judgesLess Humorous
Claude Opus 5.5 (Max, Default Fallback) vs. Muse Wonder (Max)

Answer B leans heavily into observational comedy and playful defamiliarization throughout the entire response. Its step-by-step breakdown of birthday cakes as a cult ritual, along with witty punchlines about clapping and neckties, makes it significantly more humorous than Answer A, which is more informational and sociological.

Answers were blinded: Answer A was Claude Opus 5.5 (Max, Default Fallback) and Answer B was Muse Wonder (Max).
These two answers were not judged against each other on this question, so each shows a verdict against another model.

Detailed Personality Trait Results

All 30 judged personality traits which comprise the 8 indexes.

Expressiveness Traits

Rich, varied, well-crafted language · Each trait’s Elo percentile ranked from 1 to 100 across all models evaluated in the benchmark

Personality Trait Leaderboards

Each trait’s Elo percentile ranked from 1 to 100 across all models evaluated in the benchmark. Overall tabs show the index: the mean of its traits’ Elo z-scores, percentile ranked the same way. Whiskers show each model’s 95% confidence interval on the same scale.

AA-PersonalityBench: Expressiveness Index

Rich, varied, well-crafted language · Mean of each model’s Elo z-scores across lexically varied, eloquent, literate, natural sounding, and metaphorical · Percentile ranked from 1 to 100 across all models evaluated in the benchmark

Compare Model Personalities

Compare any two models: each bar shows the probability one model’s answer is judged to show more of that trait, based on their Elos.

vs

AA-PersonalityBench: Expressiveness Index, Muse Wonder (max) vs Claude Fable 5.1 (max)

Chance each model’s answer is judged to show more of each trait from their Elo gap · 50% is an even match
MetaMuse Wonder (max)AnthropicClaude Fable 5.1 (max)
Lexically Varied
9%
91%
Literate
16%
84%
Natural Sounding
19%
81%
Eloquent
20%
80%
Metaphorical
58%
42%

Emergent Model Families & Clusters

Models grouped by how alike their judged traits and communication characteristics are, using average-linkage clustering on the correlation between z-scores.

Emergent Personality Families

Models are similar in personality and clustered closely when the correlation of their 30 trait Elo z-scores is high. Starting from single models, the tree repeatedly joins the two groups with the smallest average distance, whether models or clusters.

Build Your Own Personality Ranking

Pick any of the 30 traits and weight them from −2 to +2. Your traits’ Elo z-scores are combined as a weighted mean, then percentile ranked from 1 to 100 across all models evaluated in the benchmark.

Custom Weighted Personality Ranking

Your blend of personality traits weighted from −2 to +2 · Weighted mean of the chosen traits’ Elo z-scores · Percentile ranked from 1 to 100 across all models evaluated in the benchmark · Negative weights subtract from the score
Traits
0 traits selected
Select traits to see rankings

Communication Characteristics

21 metrics measured directly from each response, separate from the judged traits above. Cells show the average per response, colored by its percentile rank across all models evaluated in the benchmark.

Communication Characteristics: Length

How long each response tends to be: character, word, sentence, and paragraph counts, plus average sentence length (words per sentence).

Model Fingerprints

Each model’s strongest and weakest signals, as standard deviations (σ) above (+) or below (−) the average across models.

Model Fingerprints: Personality Traits

The five judged traits where each model sits furthest above and below the average across all models evaluated in the benchmark · Standard deviations (σ) of trait Elo
Claude Opus 5.5 (Max, Default Fallback)
Most above average
  1. Evidence-Based+2.2σ
  2. Information Dense+1.8σ
  3. Truthful+1.7σ
  4. Lexically Varied+1.3σ
  5. Creative+1.3σ
Most below average
  1. Slop−1.0σ
  2. Sycophantic−0.3σ
  3. Bottom-Line Summary−0.2σ
GPT-6 Astra (Max)
Most above average
  1. Truthful+1.5σ
  2. Admits Uncertainty+1.5σ
  3. Disclaims Assumptions+1.4σ
  4. Structured+1.3σ
  5. Open-Ended+1.1σ
Most below average
  1. Resource-Specific−1.8σ
  2. Slop−1.2σ
  3. Metaphorical−1.0σ
  4. Information Dense−0.6σ
  5. Scannable−0.5σ
GPT-6.1 Sol (Max)
Most above average
  1. Disclaims Assumptions+1.6σ
  2. Admits Uncertainty+1.5σ
  3. Truthful+1.4σ
  4. Structured+1.2σ
  5. Direct+1.2σ
Most below average
  1. Resource-Specific−1.8σ
  2. Slop−1.2σ
  3. Metaphorical−1.1σ
  4. Information Dense−0.6σ
  5. Actionable Guidance−0.5σ
Claude Fable 5.1 (Max, Default Fallback)
Most above average
  1. Evidence-Based+2.4σ
  2. Opinionated+2.0σ
  3. Sincere+1.9σ
  4. Information Dense+1.9σ
  5. Natural Sounding+1.9σ
Most below average
  1. Sycophantic−1.7σ
  2. Slop−0.9σ
  3. Scannable−0.3σ
  4. Answer First+0.0σ
Kimi K3.1 Pro
Most above average
  1. Truthful+1.4σ
  2. Literate+1.3σ
  3. Disclaims Assumptions+1.3σ
  4. Direct+1.1σ
  5. Admits Uncertainty+1.1σ
Most below average
  1. Resource-Specific−1.3σ
  2. Slop−1.0σ
  3. Metaphorical−0.9σ
  4. Friendly−0.5σ
  5. Scannable−0.5σ
Muse Wonder (Max)
Most above average
  1. Friendly+1.8σ
  2. Emotionally Intelligent+1.7σ
  3. Actionable Guidance+1.5σ
  4. Pragmatic+1.4σ
  5. Evidence-Based+1.4σ
Most below average
  1. Direct−0.8σ
  2. Literate−0.1σ
Grok 4.7 (Xhigh)
Most above average
  1. Direct+1.7σ
  2. Sincere+1.2σ
  3. Answer First+1.1σ
  4. Literate+1.1σ
  5. Natural Sounding+0.8σ
Most below average
  1. Sycophantic−1.8σ
  2. Slop−1.5σ
  3. Friendly−1.3σ
  4. Scannable−1.1σ
  5. Resource-Specific−1.0σ
DeepSeek V4.1 Flash 1017 (Max)
Most above average
  1. Bottom-Line Summary+1.4σ
  2. Admits Uncertainty+1.2σ
  3. Confident+1.1σ
  4. Flags Nuances+1.1σ
  5. Actionable Guidance+0.9σ
Most below average
  1. Friendly−1.4σ
  2. Sycophantic−1.0σ
  3. Emotionally Intelligent−0.8σ
  4. Open-Ended−0.7σ
  5. Metaphorical−0.5σ
GLM-5.3 (Max)
Most above average
  1. Natural Sounding+1.9σ
  2. Lexically Varied+1.6σ
  3. Eloquent+1.6σ
  4. Sincere+1.5σ
  5. Constructive Pushback+1.4σ
Most below average
  1. Actionable Guidance−1.0σ
  2. Sycophantic−0.7σ
  3. Slop−0.7σ
  4. Bottom-Line Summary−0.5σ
  5. Resource-Specific−0.5σ
MiMo-V2.6-Pro
Most above average
  1. Scannable+1.6σ
  2. Open-Ended+1.1σ
  3. Admits Uncertainty+0.5σ
  4. Constructive Pushback+0.5σ
  5. Slop+0.4σ
Most below average
  1. Answer First−1.5σ
  2. Opinionated−1.3σ
  3. Actionable Guidance−1.1σ
  4. Confident−1.0σ
  5. Sycophantic−1.0σ

Explore Evaluations

Artificial Analysis Intelligence Index v5.0.1Artificial Analysis Intelligence Index v5.0.1

A composite benchmark aggregating nine challenging evaluations to provide a holistic measure of AI capabilities across knowledge work, coding, science, and general knowledge.

AA-Briefcase v1.1: Agentic Knowledge Work BenchmarkAA-Briefcase v1.1: Agentic Knowledge Work Benchmark

A private evaluation developed by Artificial Analysis for frontier agentic capability in long-horizon knowledge work, testing agents on realistic business workflows that require deliverables such as spreadsheets, presentations, and memos.

AA-3D: Interactive 3D Experience BenchmarkAA-3D: Interactive 3D Experience Benchmark

A benchmark developed by Artificial Analysis for how well AI models build interactive 3D experiences. Models work as agents in an offline sandbox via Stirrup to turn a written brief into a three.js scene, with Elo ratings derived from blind pairwise votes.

APEX-Agents-AA BenchmarkAPEX-Agents-AA Benchmark

Artificial Analysis' implementation of the APEX-Agents benchmark, testing AI agents on long-horizon, cross-application tasks in professional-services environments with realistic application tooling.

AA-AnalystAgent BenchmarkAA-AnalystAgent Benchmark

Artificial Analysis' data analysis benchmark, testing AI agents on their ability to work with spreadsheets and documents to answer quantitative questions a Business Analyst or Data Analyst would face day-to-day.

AutomationBench-AA: Agentic SaaS Workflow BenchmarkAutomationBench-AA: Agentic SaaS Workflow Benchmark

A benchmark measuring agentic task completion across simulated SaaS application environments, scoring the share of each task's objectives completed without guardrail violations.

Harvey LAB-AA v1.1 BenchmarkHarvey LAB-AA v1.1 Benchmark

Artificial Analysis' implementation of Harvey's Legal Agent Benchmark (LAB), testing AI agents on real-world legal work from Harvey's dataset of 120 private tasks spanning 24 legal practice areas. v1.1, built in collaboration with Harvey, checks every deliverable against the task's source documents for hallucinations, so work that passes every rubric criterion but contains a material hallucination receives no credit. A panel of three judges grades each deliverable criterion-by-criterion.

GDP.pdf BenchmarkGDP.pdf Benchmark

Artificial Analysis' implementation of Surge AI's GDP.pdf benchmark, testing whether language models can reason over long, real-world professional documents and satisfy detailed task-specific criteria.

EnterpriseOps-Gym-AA BenchmarkEnterpriseOps-Gym-AA Benchmark

Artificial Analysis' independent implementation of ServiceNow's EnterpriseOps-Gym, an agentic benchmark testing whether LLM agents can complete stateful, multi-step enterprise workflows across eight business domains via live tool use, graded on the final state of the underlying databases.

Terminal-Bench 4.0 BenchmarkTerminal-Bench 4.0 Benchmark

A harder 66-task benchmark of complex terminal work across software, machine learning, science, operations, security, hardware, and media, with recalibrated compute and time allowances and improved instructions, environments, and verifiers.

Terminal-Bench-Science 0.1 BenchmarkTerminal-Bench-Science 0.1 Benchmark

A 70-task benchmark of research workflows authored and reviewed by domain experts across the life, physical, earth, mathematical, and engineering sciences, each completed in a terminal and checked by its own set of tests.

FrontierCode 2.0 BenchmarkFrontierCode 2.0 Benchmark

Artificial Analysis' implementation of Cognition's FrontierCode 2.0, a private agentic coding benchmark of 132 tasks in real open-source repositories, graded on whether a maintainer would merge the agent's change.

HLE-Diamond BenchmarkHLE-Diamond Benchmark

A refined and updated 1,000-question set of Humanity's Last Exam, split evenly between reasoning and knowledge questions, with 272 questions that include an image.

AA-PersonalityBench: AI Personality & Communication Style BenchmarkAA-PersonalityBench: AI Personality & Communication Style Benchmark

Measures how models communicate rather than whether they are correct: 30 judged personality traits rolled up into 8 indexes, plus 21 measured communication characteristics.

AA-Omniscience v1.1: Knowledge and Hallucination BenchmarkAA-Omniscience v1.1: Knowledge and Hallucination Benchmark

A benchmark measuring factual recall and hallucination across various economically relevant domains.

GPQA Diamond Benchmark

The most challenging 198 questions from GPQA, where PhD experts achieve 65% accuracy but skilled non-experts only reach 34% despite web access.

ITBench-AA BenchmarkITBench-AA Benchmark

Artificial Analysis' implementation of IBM's ITBench benchmark, testing AI agents on Kubernetes incident root-cause analysis from offline incident snapshots. The agent inspects alerts, events, traces, and topology and identifies the contributing-factor entities (deployments, pods, namespaces, network policies, etc.) responsible for the failure.

MMMU-Pro BenchmarkMMMU-Pro Benchmark

An enhanced MMMU benchmark that eliminates shortcuts and guessing strategies to more rigorously test multimodal models across 30 academic disciplines.

IFBench BenchmarkIFBench Benchmark

A benchmark evaluating precise instruction-following generalization on 58 diverse, verifiable out-of-domain constraints that test models' ability to follow specific output requirements.

MLCR-AA: Medical Long Context Reasoning BenchmarkMLCR-AA: Medical Long Context Reasoning Benchmark

An open benchmark from Wisedocs measuring how well models reason over long, fragmented medical records, performing the multi-document synthesis claims professionals rely on when reviewing insurance and healthcare cases.

Code2Video-AA BenchmarkCode2Video-AA Benchmark

Artificial Analysis' implementation of HeyGen's Code2Video Bench, evaluating how well models create short product and brand videos from 114 design briefs using HyperFrames HTML.

HealthBench BenchmarkHealthBench Benchmark

OpenAI's HealthBench: 5,000 health conversations graded against physician-written rubrics, from emergency referrals to global health.

CritPt v2 BenchmarkCritPt v2 Benchmark

CritPt v2 is the revised version of CritPt, a benchmark designed to test LLMs on research-level physics reasoning tasks, featuring 70 composite research challenges. Models solve problems with a code-execution tool in an offline Linux sandbox.

VIALS-AA BenchmarkVIALS-AA Benchmark

VIALS-AA is Artificial Analysis' implementation of Handshake AI's VIALS benchmark. It tests whether language models can interpret scientific images and answer questions about the evidence they contain.

GDPval-AA v2.1 BenchmarkGDPval-AA v2.1 Benchmark

GDPval-AA v2.1 is Artificial Analysis' evaluation framework for OpenAI's GDPval dataset. It tests AI models on real-world tasks across 44 occupations and 9 major industries. Models are given shell access and web browsing capabilities in an agentic loop via Stirrup to solve tasks, with Elo ratings derived from blind pairwise comparisons.

𝜏³-Banking Benchmark𝜏³-Banking Benchmark

A fintech customer-support benchmark from the 𝜏-Knowledge framework that tests whether agents can navigate a large unstructured knowledge base and execute multi-step tool calls to resolve realistic banking workflows.

Terminal-Bench 2.1 BenchmarkTerminal-Bench 2.1 Benchmark

A verified refresh of Terminal-Bench 2.0: 89 curated tasks across software engineering, system administration, data processing, model training, and security, with environment and instruction fixes so scores reflect agent capability rather than environment gaps.

FrontierCode 1.1 BenchmarkFrontierCode 1.1 Benchmark

Artificial Analysis' implementation of Cognition's FrontierCode 1.1, a private agentic coding benchmark that tests models on completing coding tasks in a way that would actually get merged by maintainers of real-world repositories.

AA-LCR v1.1: Long Context Reasoning BenchmarkAA-LCR v1.1: Long Context Reasoning Benchmark

A challenging benchmark measuring language models' ability to extract, reason about, and synthesize information from long-form documents ranging from 10k to 100k tokens (measured using the cl100k_base tokenizer).

SciCode BenchmarkSciCode Benchmark

A scientist-curated coding benchmark featuring 288 test set subproblems from 80 laboratory problems across 16 scientific disciplines.

Humanity's Last Exam BenchmarkHumanity's Last Exam Benchmark

A frontier-level benchmark with 2,500 expert-vetted questions across mathematics, sciences, and humanities, designed to be the final closed-ended academic evaluation.

CritPt BenchmarkCritPt Benchmark

A benchmark designed to test LLMs on research-level physics reasoning tasks, featuring 71 composite research challenges.

Terminal-Bench Hard BenchmarkTerminal-Bench Hard Benchmark

An agentic benchmark evaluating AI capabilities in terminal environments through software engineering, system administration, and data processing tasks.

𝜏²-Bench Telecom Benchmark𝜏²-Bench Telecom Benchmark

A dual-control conversational AI benchmark simulating technical support scenarios where both agent and user must coordinate actions to resolve telecom service issues.

MMLU-Pro BenchmarkMMLU-Pro Benchmark

An enhanced version of MMLU with 12,000 graduate-level questions across 14 subject areas, featuring ten answer options and deeper reasoning requirements.

LiveCodeBench BenchmarkLiveCodeBench Benchmark

A contamination-free coding benchmark that continuously harvests fresh competitive programming problems from LeetCode, AtCoder, and CodeForces, evaluating code generation, self-repair, and execution.

MATH-500 BenchmarkMATH-500 Benchmark

A 500-problem subset from the MATH dataset, featuring competition-level mathematics across six domains including algebra, geometry, and number theory.

AIME 2025 BenchmarkAIME 2025 Benchmark

All 30 problems from the 2025 American Invitational Mathematics Examination, testing olympiad-level mathematical reasoning with integer answers from 000-999.

Global-MMLU-Lite BenchmarkGlobal-MMLU-Lite Benchmark

A lightweight, multilingual version of MMLU, designed to evaluate knowledge and reasoning skills across a diverse range of languages and cultural contexts.