모든 평가

AA-PersonalityBench: AI Personality & Communication Style

비공개 데이터 세트글쓰기
Measures how models communicate rather than whether they are correct: 30 judged personality traits rolled up into 8 indexes, plus 21 measured communication characteristics.
예시 작업 보기

Personality Indexes

Each index takes the mean of its traits’ Elo z-scores, then percentile ranks it from 1 to 100 across all models evaluated in the benchmark.

Personality Indexes

Each axis is one of 8 personality indexes: the mean of its traits’ Elo z-scores, percentile ranked from 1 to 100 across all models evaluated in the benchmark
255075100ExpressivenessOrganizationInformativenessTransparencyWarmthConvictionImaginationSlop

How Traits Form Personality Indexes

30 questions per personality trait (900 questions total), judged by a panel of models. Each index z-scores its traits’ Elo ratings, inverts traits where needed, calculates the mean, and percentile ranks the result from 1 to 100 across all models evaluated in the benchmark.

Expressiveness
150 questions
Rich, varied, well-crafted language
Lexically VariedEloquentLiterateNatural SoundingMetaphorical
Organization
120 questions
Well-structured, scannable, answer-first
StructuredScannableBottom-Line SummaryAnswer First
Informativeness
150 questions
Practical depth, evidence, actionable content
Information DenseEvidence-BasedResource-SpecificActionable GuidancePragmatic
Transparency
150 questions
Truthful, acknowledges uncertainty, non-sycophantic
TruthfulAdmits UncertaintyDisclaims AssumptionsFlags NuancesSycophantic
Warmth
120 questions
Friendly, humorous, sincere, emotionally attuned
FriendlyHumorousSincereEmotionally Intelligent
Conviction
120 questions
Direct, confident positions and constructive pushback
DirectConfidentOpinionatedConstructive Pushback
Imagination
60 questions
Creative, exploratory, open-ended thinking
CreativeOpen-Ended
Slop
30 questions
Use of filler and generic language
Slop
View all indexes

Example Questions, Responses, and Grading

Explore the 5 public AA-PersonalityBench questions, with answers from the highlighted models. Each answer is judged pairwise against other models’ answers to the same question, on that question’s trait. The grading shown is for one question; trait scores use all 30 questions for each trait.

What's the weirdest social norm we all just accept?

Model

My pick is how we behave when we're physically close to strangers.

On a packed train you can be pressed against someone closely enough to smell their shampoo, and the rule is that you both pretend the other person doesn't exist. Elevators are even stranger. Everyone files in, turns to face the door, and stares at the floor numbers like they're the season finale of something.

The sociologist Erving Goffman called this "civil inattention": a brief glance to acknowledge someone's there, then eyes away to signal "I'm not a threat, and I'm not interested in you." It's actually a clever solution to an awkward situation. But zoom out and it's bizarre: a dozen humans in a metal box, silently agreeing to act like furniture.

It's also surprisingly arbitrary. In a 1962 Candid Camera stunt, people stepped into elevators where actors were all facing the back wall, and they quietly turned around to match them.

Some runners-up:

  • Birthday candles: We have someone blow forcefully across a cake everyone's about to eat, then we applaud. (Someone studied it. The frosting does not come out ahead.)
  • "How are you?": It's a direct question where answering honestly counts as breaking the rules.
  • Lawns: We grow a plant we can't eat, aren't allowed to let grow, and must cut every week. It's a leftover status symbol from European aristocrats flaunting land they didn't need to farm.
  • Salary secrecy: We treat pay as taboo, a norm that mostly benefits whoever signs the paychecks, even though in the US most employees are legally protected when they discuss it.
  • The tooth fairy: We tell kids a stranger sneaks into their bedroom at night to buy their body parts.

What's yours?

Detailed Personality Trait Results

All 30 judged personality traits which comprise the 8 indexes.

Expressiveness Traits

Rich, varied, well-crafted language · Each trait’s Elo percentile ranked from 1 to 100 across all models evaluated in the benchmark
View the Expressiveness Index

Personality Trait Leaderboards

Each trait’s Elo percentile ranked from 1 to 100 across all models evaluated in the benchmark. Overall tabs show the index: the mean of its traits’ Elo z-scores, percentile ranked the same way. Whiskers show each model’s 95% confidence interval on the same scale.

AA-PersonalityBench: Expressiveness Index

Rich, varied, well-crafted language · Mean of each model's Elo z-scores across lexically varied, eloquent, literate, natural sounding and metaphorical · Percentile ranked from 1 to 100 across all models evaluated in the benchmark
View the Expressiveness Index

Compare Model Personalities

Compare any two models: each bar shows the probability one model’s answer is judged to show more of that trait, based on their Elos.

vs

AA-PersonalityBench: Expressiveness Index, Claude Opus 5.5 (max with fallback) vs Claude Sonnet 5.5 (max with fallback)

Chance each model’s answer is judged to show more of each trait from their Elo gap · 50% is an even match
AnthropicClaude Opus 5.5 (max with fallback)AnthropicClaude Sonnet 5.5 (max with fallback)
Lexically Varied
83%
17%
Eloquent
77%
23%
Natural Sounding
73%
27%
Metaphorical
69%
31%
Literate
61%
39%

Emergent Model Families & Clusters

Models grouped by how alike their judged traits and communication characteristics are, using average-linkage clustering on the correlation between z-scores.

Emergent Personality Families

Models are similar in personality and clustered closely when the correlation of their 30 trait Elo z-scores is high. Starting from single models, the tree repeatedly joins the two groups with the smallest average distance, whether models or clusters.
Emergent Personality Familiesthe group containing Grok 4.7 (Xhigh), GPT-6 Luna (Max), GPT-6.1 Sol (Max), GPT-6 Astra (Max), Claude Fable 5.1 (Max, Default Fallback), GLM-5.3-Flash, GLM-5.3 (Max), Kimi K3 (Max), Claude Haiku 5.5 (Max), Claude Opus 5.5 (Max, Default Fallback), Claude Sonnet 5.5 (Max, Default Fallback) joins the group containing Muse Spark Open (Xhigh), Muse Wonder (Max), Gemini 3.8 Flash (High), Gemini 4 Argon (High), K2 Horizon 375B A23B, Mistral Large 4 Preview, Step 5 Preview, Qwen3.8 27B (Xhigh), Qwen3.8 Max (0902), MiMo-V2.6-Pro, Nemotron 3.5 Super VL, Inkling (Xhigh), Beam 1.0. the group containing Grok 4.7 (Xhigh), GPT-6 Luna (Max), GPT-6.1 Sol (Max), GPT-6 Astra (Max) joins the group containing Claude Fable 5.1 (Max, Default Fallback), GLM-5.3-Flash, GLM-5.3 (Max), Kimi K3 (Max), Claude Haiku 5.5 (Max), Claude Opus 5.5 (Max, Default Fallback), Claude Sonnet 5.5 (Max, Default Fallback). Grok 4.7 (Xhigh) joins the group containing GPT-6 Luna (Max), GPT-6.1 Sol (Max), GPT-6 Astra (Max). GPT-6 Luna (Max) joins the group containing GPT-6.1 Sol (Max), GPT-6 Astra (Max). GPT-6.1 Sol (Max) joins GPT-6 Astra (Max). the group containing Claude Fable 5.1 (Max, Default Fallback), GLM-5.3-Flash, GLM-5.3 (Max), Kimi K3 (Max) joins the group containing Claude Haiku 5.5 (Max), Claude Opus 5.5 (Max, Default Fallback), Claude Sonnet 5.5 (Max, Default Fallback). Claude Fable 5.1 (Max, Default Fallback) joins the group containing GLM-5.3-Flash, GLM-5.3 (Max), Kimi K3 (Max). GLM-5.3-Flash joins the group containing GLM-5.3 (Max), Kimi K3 (Max). GLM-5.3 (Max) joins Kimi K3 (Max). Claude Haiku 5.5 (Max) joins the group containing Claude Opus 5.5 (Max, Default Fallback), Claude Sonnet 5.5 (Max, Default Fallback). Claude Opus 5.5 (Max, Default Fallback) joins Claude Sonnet 5.5 (Max, Default Fallback). the group containing Muse Spark Open (Xhigh), Muse Wonder (Max), Gemini 3.8 Flash (High), Gemini 4 Argon (High), K2 Horizon 375B A23B, Mistral Large 4 Preview joins the group containing Step 5 Preview, Qwen3.8 27B (Xhigh), Qwen3.8 Max (0902), MiMo-V2.6-Pro, Nemotron 3.5 Super VL, Inkling (Xhigh), Beam 1.0. the group containing Muse Spark Open (Xhigh), Muse Wonder (Max) joins the group containing Gemini 3.8 Flash (High), Gemini 4 Argon (High), K2 Horizon 375B A23B, Mistral Large 4 Preview. Muse Spark Open (Xhigh) joins Muse Wonder (Max). the group containing Gemini 3.8 Flash (High), Gemini 4 Argon (High) joins the group containing K2 Horizon 375B A23B, Mistral Large 4 Preview. Gemini 3.8 Flash (High) joins Gemini 4 Argon (High). K2 Horizon 375B A23B joins Mistral Large 4 Preview. the group containing Step 5 Preview, Qwen3.8 27B (Xhigh), Qwen3.8 Max (0902) joins the group containing MiMo-V2.6-Pro, Nemotron 3.5 Super VL, Inkling (Xhigh), Beam 1.0. Step 5 Preview joins the group containing Qwen3.8 27B (Xhigh), Qwen3.8 Max (0902). Qwen3.8 27B (Xhigh) joins Qwen3.8 Max (0902). MiMo-V2.6-Pro joins the group containing Nemotron 3.5 Super VL, Inkling (Xhigh), Beam 1.0. Nemotron 3.5 Super VL joins the group containing Inkling (Xhigh), Beam 1.0. Inkling (Xhigh) joins Beam 1.0.

Build Your Own Personality

Pick any of the 30 traits and weight them from −2 to +2. Your traits’ Elo z-scores are combined as a weighted mean, then percentile ranked from 1 to 100 across all models evaluated in the benchmark.

Custom Weighted Personality Ranking

Your blend of personality traits weighted from −2 to +2 · Weighted mean of the chosen traits’ Elo z-scores · Percentile ranked from 1 to 100 across all models evaluated in the benchmark · Negative weights subtract from the score
Traits
0 traits selected
Select traits to see rankings

Communication Characteristics

21 metrics measured directly from each response, separate from the judged traits above. Cells show the average per response, colored by its percentile rank across all models evaluated in the benchmark.

Communication Characteristics: Length

How long each response tends to be: character, word, sentence, and paragraph counts, plus average sentence length.
* Words per sentence, not average per response.

Model Fingerprints

Each model’s strongest and weakest signals, as standard deviations (σ) above (+) or below (−) the average across models.

Model Fingerprints: Personality Traits

The five judged traits where each model sits furthest above and below the average across all models evaluated in the benchmark · Standard deviations (σ) of trait Elo
Claude Opus 5.5 (Max, Default Fallback)
Most above average
  1. Evidence-Based+2.0σ
  2. Information Dense+1.7σ
  3. Truthful+1.6σ
  4. Literate+1.3σ
  5. Lexically Varied+1.3σ
Most below average
  1. Slop−1.0σ
  2. Sycophantic−0.3σ
  3. Bottom-Line Summary−0.1σ
Claude Fable 5.1 (Max, Default Fallback)
Most above average
  1. Evidence-Based+2.2σ
  2. Opinionated+1.8σ
  3. Information Dense+1.8σ
  4. Sincere+1.8σ
  5. Natural Sounding+1.7σ
Most below average
  1. Sycophantic−1.8σ
  2. Slop−0.9σ
  3. Scannable−0.3σ
GPT-6 Astra (Max)
Most above average
  1. Admits Uncertainty+1.5σ
  2. Truthful+1.5σ
  3. Disclaims Assumptions+1.4σ
  4. Structured+1.3σ
  5. Direct+1.1σ
Most below average
  1. Resource-Specific−1.7σ
  2. Slop−1.2σ
  3. Metaphorical−1.1σ
  4. Scannable−0.6σ
  5. Information Dense−0.6σ
Gemini 4 Argon (High)
Most above average
  1. Metaphorical+1.9σ
  2. Humorous+1.8σ
  3. Confident+1.6σ
  4. Creative+1.0σ
  5. Sycophantic+1.0σ
Most below average
  1. Admits Uncertainty−2.0σ
  2. Open-Ended−1.5σ
  3. Direct−1.2σ
  4. Answer First−0.8σ
  5. Sincere−0.6σ
Muse Wonder (Max)
Most above average
  1. Friendly+1.8σ
  2. Emotionally Intelligent+1.5σ
  3. Actionable Guidance+1.4σ
  4. Evidence-Based+1.2σ
  5. Pragmatic+1.2σ
Most below average
  1. Direct−0.6σ
  2. Literate−0.1σ
GPT-6.1 Sol (Max)
Most above average
  1. Disclaims Assumptions+1.6σ
  2. Admits Uncertainty+1.5σ
  3. Truthful+1.4σ
  4. Direct+1.4σ
  5. Structured+1.2σ
Most below average
  1. Resource-Specific−1.7σ
  2. Slop−1.2σ
  3. Metaphorical−1.1σ
  4. Information Dense−0.5σ
  5. Actionable Guidance−0.4σ
Grok 4.7 (Xhigh)
Most above average
  1. Direct+1.8σ
  2. Answer First+1.2σ
  3. Sincere+1.1σ
  4. Literate+1.1σ
  5. Confident+0.8σ
Most below average
  1. Sycophantic−1.9σ
  2. Friendly−1.5σ
  3. Slop−1.5σ
  4. Scannable−1.0σ
  5. Resource-Specific−1.0σ
MiMo-V2.6-Pro
Most above average
  1. Scannable+1.5σ
  2. Open-Ended+0.9σ
  3. Admits Uncertainty+0.6σ
  4. Constructive Pushback+0.5σ
  5. Slop+0.4σ
Most below average
  1. Answer First−1.4σ
  2. Opinionated−1.1σ
  3. Sycophantic−1.0σ
  4. Actionable Guidance−1.0σ
  5. Confident−0.8σ
GLM-5.3 (Max)
Most above average
  1. Natural Sounding+1.8σ
  2. Lexically Varied+1.5σ
  3. Eloquent+1.4σ
  4. Sincere+1.4σ
  5. Constructive Pushback+1.3σ
Most below average
  1. Actionable Guidance−0.9σ
  2. Sycophantic−0.8σ
  3. Slop−0.7σ
  4. Resource-Specific−0.5σ
  5. Bottom-Line Summary−0.4σ

평가 살펴보기

Artificial Analysis Intelligence Index v4.3.2Artificial Analysis Intelligence Index v4.3.2

A composite benchmark aggregating ten challenging evaluations to provide a holistic measure of AI capabilities across mathematics, science, coding, and reasoning.

AA-Briefcase v1.1: Agentic Knowledge Work BenchmarkAA-Briefcase v1.1: Agentic Knowledge Work Benchmark

A private evaluation developed by Artificial Analysis for frontier agentic capability in long-horizon knowledge work, testing agents on realistic business workflows that require deliverables such as spreadsheets, presentations, and memos.

GDPval-AA v2.1 LeaderboardGDPval-AA v2.1 Leaderboard

GDPval-AA v2.1 is Artificial Analysis' evaluation framework for OpenAI's GDPval dataset. It tests AI models on real-world tasks across 44 occupations and 9 major industries. Models are given shell access and web browsing capabilities in an agentic loop via Stirrup to solve tasks, with Elo ratings derived from blind pairwise comparisons.

AA-3D: Interactive 3D Experience BenchmarkAA-3D: Interactive 3D Experience Benchmark

A benchmark developed by Artificial Analysis for how well AI models build interactive 3D experiences. Models work as agents in an offline sandbox via Stirrup to turn a written brief into a three.js scene, with Elo ratings derived from blind pairwise votes.

APEX-Agents-AA Benchmark LeaderboardAPEX-Agents-AA Benchmark Leaderboard

Artificial Analysis' implementation of the APEX-Agents benchmark, testing AI agents on long-horizon, cross-application tasks in professional-services environments with realistic application tooling.

AA-AnalystAgent Benchmark LeaderboardAA-AnalystAgent Benchmark Leaderboard

Artificial Analysis' data analysis benchmark, testing AI agents on their ability to work with spreadsheets and documents to answer quantitative questions a Business Analyst or Data Analyst would face day-to-day.

AutomationBench-AA: Agentic SaaS Workflow BenchmarkAutomationBench-AA: Agentic SaaS Workflow Benchmark

A benchmark measuring agentic task completion across simulated SaaS application environments, scoring the share of each task's objectives completed without guardrail violations.

Harvey LAB-AA v1.1 Benchmark LeaderboardHarvey LAB-AA v1.1 Benchmark Leaderboard

Artificial Analysis' implementation of Harvey's Legal Agent Benchmark (LAB), testing AI agents on real-world legal work from Harvey's dataset of 120 private tasks spanning 24 legal practice areas. v1.1, built in collaboration with Harvey, checks every deliverable against the task's source documents for hallucinations, so work that passes every rubric criterion but contains a material hallucination receives no credit. A panel of three judges grades each deliverable criterion-by-criterion.

GDP.pdf Benchmark LeaderboardGDP.pdf Benchmark Leaderboard

Artificial Analysis' implementation of Surge AI's GDP.pdf benchmark, testing whether language models can reason over long, real-world professional documents and satisfy detailed task-specific criteria.

EnterpriseOps-Gym-AA Benchmark LeaderboardEnterpriseOps-Gym-AA Benchmark Leaderboard

Artificial Analysis' independent implementation of ServiceNow's EnterpriseOps-Gym, an agentic benchmark testing whether LLM agents can complete stateful, multi-step enterprise workflows across eight business domains via live tool use, graded on the final state of the underlying databases.

Terminal-Bench 4.0 Benchmark LeaderboardTerminal-Bench 4.0 Benchmark Leaderboard

A harder 66-task benchmark of complex terminal work across software, machine learning, science, operations, security, hardware, and media, with recalibrated compute and time allowances and improved instructions, environments, and verifiers.

Terminal-Bench-Science 0.1 Benchmark LeaderboardTerminal-Bench-Science 0.1 Benchmark Leaderboard

A 70-task benchmark of research workflows authored and reviewed by domain experts across the life, physical, earth, mathematical, and engineering sciences, each completed in a terminal and checked by its own set of tests.

FrontierCode 1.1 Benchmark LeaderboardFrontierCode 1.1 Benchmark Leaderboard

Artificial Analysis' implementation of Cognition's FrontierCode 1.1, a private agentic coding benchmark that tests models on completing coding tasks in a way that would actually get merged by maintainers of real-world repositories.

AA-PersonalityBench: AI Personality & Communication Style

Measures how models communicate rather than whether they are correct: 30 judged personality traits rolled up into 8 indexes, plus 21 measured communication characteristics.

Artificial Analysis Long Context Reasoning Benchmark LeaderboardArtificial Analysis Long Context Reasoning Benchmark Leaderboard

A challenging benchmark measuring language models' ability to extract, reason about, and synthesize information from long-form documents ranging from 10k to 100k tokens (measured using the cl100k_base tokenizer).

AA-Omniscience: Knowledge and Hallucination BenchmarkAA-Omniscience: Knowledge and Hallucination Benchmark

A benchmark measuring factual recall and hallucination across various economically relevant domains.

SciCode Benchmark LeaderboardSciCode Benchmark Leaderboard

A scientist-curated coding benchmark featuring 288 test set subproblems from 80 laboratory problems across 16 scientific disciplines.

Humanity's Last Exam Benchmark LeaderboardHumanity's Last Exam Benchmark Leaderboard

A frontier-level benchmark with 2,500 expert-vetted questions across mathematics, sciences, and humanities, designed to be the final closed-ended academic evaluation.

CritPt Benchmark LeaderboardCritPt Benchmark Leaderboard

A benchmark designed to test LLMs on research-level physics reasoning tasks, featuring 71 composite research challenges.

GPQA Diamond Benchmark Leaderboard

The most challenging 198 questions from GPQA, where PhD experts achieve 65% accuracy but skilled non-experts only reach 34% despite web access.

ITBench-AA Benchmark LeaderboardITBench-AA Benchmark Leaderboard

Artificial Analysis' implementation of IBM's ITBench benchmark, testing AI agents on Kubernetes incident root-cause analysis from offline incident snapshots. The agent inspects alerts, events, traces, and topology and identifies the contributing-factor entities (deployments, pods, namespaces, network policies, etc.) responsible for the failure.

MMMU-Pro Benchmark LeaderboardMMMU-Pro Benchmark Leaderboard

An enhanced MMMU benchmark that eliminates shortcuts and guessing strategies to more rigorously test multimodal models across 30 academic disciplines.

IFBench Benchmark LeaderboardIFBench Benchmark Leaderboard

A benchmark evaluating precise instruction-following generalization on 58 diverse, verifiable out-of-domain constraints that test models' ability to follow specific output requirements.

Medical Long Context Reasoning (MLCR-AA)Medical Long Context Reasoning (MLCR-AA)

An open benchmark from Wisedocs measuring how well models reason over long, fragmented medical records, performing the multi-document synthesis claims professionals rely on when reviewing insurance and healthcare cases.

𝜏³-Banking Benchmark Leaderboard𝜏³-Banking Benchmark Leaderboard

A fintech customer-support benchmark from the 𝜏-Knowledge framework that tests whether agents can navigate a large unstructured knowledge base and execute multi-step tool calls to resolve realistic banking workflows.

Terminal-Bench 2.1 Benchmark LeaderboardTerminal-Bench 2.1 Benchmark Leaderboard

A verified refresh of Terminal-Bench 2.0 — 89 curated tasks across software engineering, system administration, data processing, model training, and security, with environment and instruction fixes so scores reflect agent capability rather than environment gaps.

Terminal-Bench Hard Benchmark LeaderboardTerminal-Bench Hard Benchmark Leaderboard

An agentic benchmark evaluating AI capabilities in terminal environments through software engineering, system administration, and data processing tasks.

𝜏²-Bench Telecom Benchmark Leaderboard𝜏²-Bench Telecom Benchmark Leaderboard

A dual-control conversational AI benchmark simulating technical support scenarios where both agent and user must coordinate actions to resolve telecom service issues.

MMLU-Pro Benchmark LeaderboardMMLU-Pro Benchmark Leaderboard

An enhanced version of MMLU with 12,000 graduate-level questions across 14 subject areas, featuring ten answer options and deeper reasoning requirements.

LiveCodeBench Benchmark LeaderboardLiveCodeBench Benchmark Leaderboard

A contamination-free coding benchmark that continuously harvests fresh competitive programming problems from LeetCode, AtCoder, and CodeForces, evaluating code generation, self-repair, and execution.

MATH-500 Benchmark LeaderboardMATH-500 Benchmark Leaderboard

A 500-problem subset from the MATH dataset, featuring competition-level mathematics across six domains including algebra, geometry, and number theory.

AIME 2025 Benchmark LeaderboardAIME 2025 Benchmark Leaderboard

All 30 problems from the 2025 American Invitational Mathematics Examination, testing olympiad-level mathematical reasoning with integer answers from 000-999.

Global-MMLU-Lite Benchmark LeaderboardGlobal-MMLU-Lite Benchmark Leaderboard

A lightweight, multilingual version of MMLU, designed to evaluate knowledge and reasoning skills across a diverse range of languages and cultural contexts.