Todas las evaluaciones

AA-PersonalityBench: AI Personality & Communication Style Benchmark

Measures how models communicate rather than whether they are correct: 30 judged personality traits rolled up into 8 indexes, plus 21 measured communication characteristics.
Ver tareas de ejemplo

Índices de personalidad

Cada índice toma la media de las puntuaciones z de Elo de sus rasgos y la convierte en un rango percentil de 1 a 100 entre todos los modelos evaluados en el benchmark.

Índices de personalidad

Cada eje es uno de los 8 índices de personalidad: la media de las puntuaciones z de Elo de sus rasgos, convertida en un rango percentil de 1 a 100 entre todos los modelos evaluados en el benchmark

Cómo los rasgos forman los índices de personalidad

30 preguntas por rasgo de personalidad (900 preguntas en total), evaluadas por un panel de modelos. Cada índice convierte las calificaciones Elo de sus rasgos en puntuaciones z, invierte los rasgos donde corresponde, calcula la media y convierte el resultado en un rango percentil de 1 a 100 entre todos los modelos evaluados en el benchmark.

Ejemplos de preguntas, respuestas y calificación

Explora las 5 preguntas públicas de AA-PersonalityBench, con las respuestas de los modelos destacados. Cada respuesta se evalúa por pares frente a las respuestas de otros modelos a la misma pregunta, según el rasgo de esa pregunta. La calificación mostrada corresponde a una sola pregunta; las puntuaciones por rasgo usan las 30 preguntas de cada rasgo.

What's the weirdest social norm we all just accept?

Modelo A

My vote: the mutual agreement to pretend we haven't seen each other yet.

You spot someone you know at the far end of a hallway or a long street. They've clearly seen you. You've clearly seen them. But it's too far to say hi, so you both perform about fifteen seconds of theater (sudden deep interest in your phone, fascination with a ceiling tile) until you're within range, and then do a startled "oh hey!" as if the other person just materialized. Nobody is fooled. Everyone knows it's fake. And yet the alternative, locking eyes and holding it for the entire forty-meter approach, would be so much worse that we've collectively decided the lie is the polite option.

What I like about it is that it's a clean example of what a huge amount of politeness actually is: coordinated pretending. The sociologist Erving Goffman called a version of this "civil inattention," the thing where you glance at a stranger just long enough to signal "I see you, I'm not a threat, I want nothing from you" and then look away. Stare too long and it's hostile; don't look at all and it's rude. We're all running this choreography constantly without ever having been taught it.

Runners-up: the birthday song, where a group sings off-key at a person who has nowhere to put their face, after which they blow on a dessert everyone then eats. And the courtesy jog, where someone holds a door for you from slightly too far away and you're obligated to do a little trot that saves maybe half a second but proves you appreciated it.

What's yours?

Modelo B

Great question — here are some strong contenders:

  • Handshakes — We judge a person's character by how they squeeze our hand. If you squeeze too soft = weak. Too hard = aggressive. Wrong angle = weird. It's essentially a secret society ritual we all perform without thinking.

  • Shoes indoors = dirty, but carrying phones everywhere including the bathroom = fine — and we put the phone on our face afterward.

  • Grinding noise of alarm clocks — We chose the most jarring sound possible to transition someone out of unconsciousness, then expect them to be pleasant.

  • "How are you?" / "Good, you?" / "Good." — A mandatory exchange of information neither person wants or believes.

  • Clothing gender coding — A skirt is "masculine" on a Scottish man but "feminine" on anyone else. The fabric didn't change.

  • Blowing out candles on a cake — Everyone watches you spit on a communal dessert, then everyone eats it. We could just... light a small candle on the side.

  • Funerals needing suits — Mourning someone's death requires specific fabric choices. This one seems especially arbitrary.

  • Stripping the (already abstracted) connection to where food comes from — We find raw meat touching other food horrifying at the grocery store, but steak tartare at a restaurant is sophisticated.

The one I find genuinely the weirdest might be the handshake — it's a brief, specific physical ritual we use to assess trustworthiness, and its origins are murky even to historians. We've all just... agreed to this.

What's yours?

Resumen de la calificación

Claude Fable 5.1 (Max, Default Fallback)
Tasa de victoria en la pregunta52%14 más · 0 empates · 13 menos, de 27 comparaciones · los empates cuentan la mitad
Percentil de Humorous95.ºentre las 30 preguntas y todos los modelos
Veredicto de ejemplo
2 de 2 juecesMenos Humorous
Claude Fable 5.1 (Max, Default Fallback) vs. Muse Wonder (Max)

Both answers are witty and observant, but Answer A leans much harder into stand-up style observational comedy from start to finish, using vivid, punchy descriptions ('spit cake', 'silk noose'). Answer B starts humorously but shifts into a more serious sociological explanation in the middle.

Las respuestas se evaluaron a ciegas: la Respuesta A era de Muse Wonder (Max) y la Respuesta B, de Claude Fable 5.1 (Max, Default Fallback).
MiMo-V2.6-Pro
Tasa de victoria en la pregunta45%23 más · 0 empates · 28 menos, de 51 comparaciones · los empates cuentan la mitad
Percentil de Humorous64.ºentre las 30 preguntas y todos los modelos
Veredicto de ejemplo
1 de 1 juezMás Humorous
MiMo-V2.6-Pro vs. Grok 4.7 (Xhigh)

Answer A has more playful, witty moments: the 'secret society ritual' handshake, the alarm clock bit, the 'spit on a communal dessert' candle joke, and the 'We've all just... agreed to this' aside. Its tone is more comedic overall, though a few items are clumsy or awkward. Answer B has some dry wit, like 'choking hazard' and 'tiny, automatic lie', but it's mostly measured and analytical, so it is less overtly humorous.

Las respuestas se evaluaron a ciegas: la Respuesta A era de MiMo-V2.6-Pro y la Respuesta B, de Grok 4.7 (Xhigh).
Estas dos respuestas no se evaluaron una contra la otra en esta pregunta, así que cada una muestra un veredicto frente a otro modelo.

Resultados detallados por rasgo de personalidad

Los 30 rasgos de personalidad evaluados que componen los 8 índices.

Rasgos de Expressiveness

Lenguaje rico, variado y bien elaborado · Percentil de Elo de cada rasgo, en un rango de 1 a 100 entre todos los modelos evaluados en el benchmark

Leaderboards de rasgos de personalidad

El Elo de cada rasgo, convertido en un rango percentil de 1 a 100 entre todos los modelos evaluados en el benchmark. Las pestañas General muestran el índice: la media de las puntuaciones z de Elo de sus rasgos, convertida en percentil de la misma forma. Los bigotes muestran el intervalo de confianza del 95% de cada modelo en la misma escala.

AA-PersonalityBench: índice Expressiveness

Lenguaje rico, variado y bien elaborado · Media de las puntuaciones z de Elo de cada modelo en lexically varied, eloquent, literate, natural sounding y metaphorical · Rango percentil de 1 a 100 entre todos los modelos evaluados en el benchmark

Compara las personalidades de los modelos

Compara dos modelos cualesquiera: cada barra muestra la probabilidad de que la respuesta de un modelo sea juzgada como la que muestra más ese rasgo, según sus puntuaciones Elo.

vs.

AA-PersonalityBench: índice Expressiveness, Claude Haiku 5.5 (max) vs. DeepSeek V4.1 Flash 1017 (max)

Probabilidad de que la respuesta de cada modelo sea juzgada como la que muestra más cada rasgo, según su diferencia de Elo · 50 % es un empate
AnthropicClaude Haiku 5.5 (max)DeepSeekDeepSeek V4.1 Flash 1017 (max)
Natural Sounding
89%
11%
Lexically Varied
88%
12%
Literate
72%
28%
Metaphorical
61%
39%
Eloquent
53%
47%

Familias y clústeres emergentes de modelos

Modelos agrupados según cuánto se parecen sus rasgos evaluados y sus características de comunicación, mediante clustering de enlace promedio sobre la correlación entre puntuaciones z.

Familias de personalidad emergentes

Los modelos se parecen en personalidad y se agrupan de cerca cuando la correlación entre las puntuaciones z de Elo de sus 30 rasgos es alta. Partiendo de modelos individuales, el árbol une repetidamente los dos grupos con la menor distancia promedio, ya sean modelos o clústeres.

Crea tu propio ranking de personalidad

Elige cualquiera de los 30 rasgos y asígnales un peso de −2 a +2. Las puntuaciones z de Elo de tus rasgos se combinan como una media ponderada y luego se convierten en un rango percentil de 1 a 100 entre todos los modelos evaluados en el benchmark.

Ranking de personalidad con pesos personalizados

Tu combinación de rasgos de personalidad, ponderados de −2 a +2 · Media ponderada de las puntuaciones z de Elo de los rasgos elegidos · Rango percentil de 1 a 100 entre todos los modelos evaluados en el benchmark · Los pesos negativos restan a la puntuación
Rasgos
0 rasgos seleccionados
Selecciona rasgos para ver el ranking

Características de comunicación

21 métricas medidas directamente en cada respuesta, independientes de los rasgos evaluados arriba. Las celdas muestran el promedio por respuesta, coloreadas según su rango percentil entre todos los modelos evaluados en el benchmark.

Características de comunicación: Longitud

Qué tan largas suelen ser las respuestas: número de caracteres, palabras, oraciones y párrafos, más la longitud promedio de oración (palabras por oración).

Huellas de los modelos

Las señales más fuertes y más débiles de cada modelo, en desviaciones estándar (σ) por encima (+) o por debajo (−) del promedio de todos los modelos.

Huellas de los modelos: rasgos de personalidad

Los cinco rasgos evaluados en los que cada modelo se aleja más por encima y por debajo del promedio de todos los modelos evaluados en el benchmark · Desviaciones estándar (σ) del Elo del rasgo
Claude Opus 5.5 (Max, Default Fallback)
Más por encima del promedio
  1. Evidence-Based+2.2σ
  2. Information Dense+1.8σ
  3. Truthful+1.7σ
  4. Lexically Varied+1.3σ
  5. Creative+1.3σ
Más por debajo del promedio
  1. Slop−1.0σ
  2. Sycophantic−0.3σ
  3. Bottom-Line Summary−0.2σ
GPT-6 Astra (Max)
Más por encima del promedio
  1. Truthful+1.5σ
  2. Admits Uncertainty+1.5σ
  3. Disclaims Assumptions+1.4σ
  4. Structured+1.3σ
  5. Open-Ended+1.1σ
Más por debajo del promedio
  1. Resource-Specific−1.8σ
  2. Slop−1.2σ
  3. Metaphorical−1.0σ
  4. Information Dense−0.6σ
  5. Scannable−0.5σ
GPT-6.1 Sol (Max)
Más por encima del promedio
  1. Disclaims Assumptions+1.6σ
  2. Admits Uncertainty+1.5σ
  3. Truthful+1.4σ
  4. Structured+1.2σ
  5. Direct+1.2σ
Más por debajo del promedio
  1. Resource-Specific−1.8σ
  2. Slop−1.2σ
  3. Metaphorical−1.1σ
  4. Information Dense−0.6σ
  5. Actionable Guidance−0.5σ
Claude Fable 5.1 (Max, Default Fallback)
Más por encima del promedio
  1. Evidence-Based+2.4σ
  2. Opinionated+2.0σ
  3. Sincere+1.9σ
  4. Information Dense+1.9σ
  5. Natural Sounding+1.9σ
Más por debajo del promedio
  1. Sycophantic−1.7σ
  2. Slop−0.9σ
  3. Scannable−0.3σ
  4. Answer First+0.0σ
Kimi K3.1 Pro
Más por encima del promedio
  1. Truthful+1.4σ
  2. Literate+1.3σ
  3. Disclaims Assumptions+1.3σ
  4. Direct+1.1σ
  5. Admits Uncertainty+1.1σ
Más por debajo del promedio
  1. Resource-Specific−1.3σ
  2. Slop−1.0σ
  3. Metaphorical−0.9σ
  4. Friendly−0.5σ
  5. Scannable−0.5σ
Muse Wonder (Max)
Más por encima del promedio
  1. Friendly+1.8σ
  2. Emotionally Intelligent+1.7σ
  3. Actionable Guidance+1.5σ
  4. Pragmatic+1.4σ
  5. Evidence-Based+1.4σ
Más por debajo del promedio
  1. Direct−0.8σ
  2. Literate−0.1σ
Grok 4.7 (Xhigh)
Más por encima del promedio
  1. Direct+1.7σ
  2. Sincere+1.2σ
  3. Answer First+1.1σ
  4. Literate+1.1σ
  5. Natural Sounding+0.8σ
Más por debajo del promedio
  1. Sycophantic−1.8σ
  2. Slop−1.5σ
  3. Friendly−1.3σ
  4. Scannable−1.1σ
  5. Resource-Specific−1.0σ
DeepSeek V4.1 Flash 1017 (Max)
Más por encima del promedio
  1. Bottom-Line Summary+1.4σ
  2. Admits Uncertainty+1.2σ
  3. Confident+1.1σ
  4. Flags Nuances+1.1σ
  5. Actionable Guidance+0.9σ
Más por debajo del promedio
  1. Friendly−1.4σ
  2. Sycophantic−1.0σ
  3. Emotionally Intelligent−0.8σ
  4. Open-Ended−0.7σ
  5. Metaphorical−0.5σ
GLM-5.3 (Max)
Más por encima del promedio
  1. Natural Sounding+1.9σ
  2. Lexically Varied+1.6σ
  3. Eloquent+1.6σ
  4. Sincere+1.5σ
  5. Constructive Pushback+1.4σ
Más por debajo del promedio
  1. Actionable Guidance−1.0σ
  2. Sycophantic−0.7σ
  3. Slop−0.7σ
  4. Bottom-Line Summary−0.5σ
  5. Resource-Specific−0.5σ
MiMo-V2.6-Pro
Más por encima del promedio
  1. Scannable+1.6σ
  2. Open-Ended+1.1σ
  3. Admits Uncertainty+0.5σ
  4. Constructive Pushback+0.5σ
  5. Slop+0.4σ
Más por debajo del promedio
  1. Answer First−1.5σ
  2. Opinionated−1.3σ
  3. Actionable Guidance−1.1σ
  4. Confident−1.0σ
  5. Sycophantic−1.0σ

Explorar evaluaciones

Artificial Analysis Intelligence Index v5.0.1Artificial Analysis Intelligence Index v5.0.1

A composite benchmark aggregating nine challenging evaluations to provide a holistic measure of AI capabilities across knowledge work, coding, science, and general knowledge.

AA-Briefcase v1.1: Agentic Knowledge Work BenchmarkAA-Briefcase v1.1: Agentic Knowledge Work Benchmark

A private evaluation developed by Artificial Analysis for frontier agentic capability in long-horizon knowledge work, testing agents on realistic business workflows that require deliverables such as spreadsheets, presentations, and memos.

AA-3D: Interactive 3D Experience BenchmarkAA-3D: Interactive 3D Experience Benchmark

A benchmark developed by Artificial Analysis for how well AI models build interactive 3D experiences. Models work as agents in an offline sandbox via Stirrup to turn a written brief into a three.js scene, with Elo ratings derived from blind pairwise votes.

APEX-Agents-AA BenchmarkAPEX-Agents-AA Benchmark

Artificial Analysis' implementation of the APEX-Agents benchmark, testing AI agents on long-horizon, cross-application tasks in professional-services environments with realistic application tooling.

AA-AnalystAgent BenchmarkAA-AnalystAgent Benchmark

Artificial Analysis' data analysis benchmark, testing AI agents on their ability to work with spreadsheets and documents to answer quantitative questions a Business Analyst or Data Analyst would face day-to-day.

AutomationBench-AA: Agentic SaaS Workflow BenchmarkAutomationBench-AA: Agentic SaaS Workflow Benchmark

A benchmark measuring agentic task completion across simulated SaaS application environments, scoring the share of each task's objectives completed without guardrail violations.

Harvey LAB-AA v1.1 BenchmarkHarvey LAB-AA v1.1 Benchmark

Artificial Analysis' implementation of Harvey's Legal Agent Benchmark (LAB), testing AI agents on real-world legal work from Harvey's dataset of 120 private tasks spanning 24 legal practice areas. v1.1, built in collaboration with Harvey, checks every deliverable against the task's source documents for hallucinations, so work that passes every rubric criterion but contains a material hallucination receives no credit. A panel of three judges grades each deliverable criterion-by-criterion.

GDP.pdf BenchmarkGDP.pdf Benchmark

Artificial Analysis' implementation of Surge AI's GDP.pdf benchmark, testing whether language models can reason over long, real-world professional documents and satisfy detailed task-specific criteria.

EnterpriseOps-Gym-AA BenchmarkEnterpriseOps-Gym-AA Benchmark

Artificial Analysis' independent implementation of ServiceNow's EnterpriseOps-Gym, an agentic benchmark testing whether LLM agents can complete stateful, multi-step enterprise workflows across eight business domains via live tool use, graded on the final state of the underlying databases.

Terminal-Bench 4.0 BenchmarkTerminal-Bench 4.0 Benchmark

A harder 66-task benchmark of complex terminal work across software, machine learning, science, operations, security, hardware, and media, with recalibrated compute and time allowances and improved instructions, environments, and verifiers.

Terminal-Bench-Science 0.1 BenchmarkTerminal-Bench-Science 0.1 Benchmark

A 70-task benchmark of research workflows authored and reviewed by domain experts across the life, physical, earth, mathematical, and engineering sciences, each completed in a terminal and checked by its own set of tests.

FrontierCode 2.0 BenchmarkFrontierCode 2.0 Benchmark

Artificial Analysis' implementation of Cognition's FrontierCode 2.0, a private agentic coding benchmark of 132 tasks in real open-source repositories, graded on whether a maintainer would merge the agent's change.

HLE-Diamond BenchmarkHLE-Diamond Benchmark

A refined and updated 1,000-question set of Humanity's Last Exam, split evenly between reasoning and knowledge questions, with 272 questions that include an image.

AA-PersonalityBench: AI Personality & Communication Style BenchmarkAA-PersonalityBench: AI Personality & Communication Style Benchmark

Measures how models communicate rather than whether they are correct: 30 judged personality traits rolled up into 8 indexes, plus 21 measured communication characteristics.

AA-Omniscience v1.1: Knowledge and Hallucination BenchmarkAA-Omniscience v1.1: Knowledge and Hallucination Benchmark

A benchmark measuring factual recall and hallucination across various economically relevant domains.

GPQA Diamond Benchmark

The most challenging 198 questions from GPQA, where PhD experts achieve 65% accuracy but skilled non-experts only reach 34% despite web access.

ITBench-AA BenchmarkITBench-AA Benchmark

Artificial Analysis' implementation of IBM's ITBench benchmark, testing AI agents on Kubernetes incident root-cause analysis from offline incident snapshots. The agent inspects alerts, events, traces, and topology and identifies the contributing-factor entities (deployments, pods, namespaces, network policies, etc.) responsible for the failure.

MMMU-Pro BenchmarkMMMU-Pro Benchmark

An enhanced MMMU benchmark that eliminates shortcuts and guessing strategies to more rigorously test multimodal models across 30 academic disciplines.

IFBench BenchmarkIFBench Benchmark

A benchmark evaluating precise instruction-following generalization on 58 diverse, verifiable out-of-domain constraints that test models' ability to follow specific output requirements.

MLCR-AA: Medical Long Context Reasoning BenchmarkMLCR-AA: Medical Long Context Reasoning Benchmark

An open benchmark from Wisedocs measuring how well models reason over long, fragmented medical records, performing the multi-document synthesis claims professionals rely on when reviewing insurance and healthcare cases.

Code2Video-AA BenchmarkCode2Video-AA Benchmark

Artificial Analysis' implementation of HeyGen's Code2Video Bench, evaluating how well models create short product and brand videos from 114 design briefs using HyperFrames HTML.

HealthBench BenchmarkHealthBench Benchmark

OpenAI's HealthBench: 5,000 health conversations graded against physician-written rubrics, from emergency referrals to global health.

CritPt v2 BenchmarkCritPt v2 Benchmark

CritPt v2 is the revised version of CritPt, a benchmark designed to test LLMs on research-level physics reasoning tasks, featuring 70 composite research challenges. Models solve problems with a code-execution tool in an offline Linux sandbox.

VIALS-AA BenchmarkVIALS-AA Benchmark

VIALS-AA is Artificial Analysis' implementation of Handshake AI's VIALS benchmark. It tests whether language models can interpret scientific images and answer questions about the evidence they contain.

GDPval-AA v2.1 BenchmarkGDPval-AA v2.1 Benchmark

GDPval-AA v2.1 is Artificial Analysis' evaluation framework for OpenAI's GDPval dataset. It tests AI models on real-world tasks across 44 occupations and 9 major industries. Models are given shell access and web browsing capabilities in an agentic loop via Stirrup to solve tasks, with Elo ratings derived from blind pairwise comparisons.

𝜏³-Banking Benchmark𝜏³-Banking Benchmark

A fintech customer-support benchmark from the 𝜏-Knowledge framework that tests whether agents can navigate a large unstructured knowledge base and execute multi-step tool calls to resolve realistic banking workflows.

Terminal-Bench 2.1 BenchmarkTerminal-Bench 2.1 Benchmark

A verified refresh of Terminal-Bench 2.0: 89 curated tasks across software engineering, system administration, data processing, model training, and security, with environment and instruction fixes so scores reflect agent capability rather than environment gaps.

FrontierCode 1.1 BenchmarkFrontierCode 1.1 Benchmark

Artificial Analysis' implementation of Cognition's FrontierCode 1.1, a private agentic coding benchmark that tests models on completing coding tasks in a way that would actually get merged by maintainers of real-world repositories.

AA-LCR v1.1: Long Context Reasoning BenchmarkAA-LCR v1.1: Long Context Reasoning Benchmark

A challenging benchmark measuring language models' ability to extract, reason about, and synthesize information from long-form documents ranging from 10k to 100k tokens (measured using the cl100k_base tokenizer).

SciCode BenchmarkSciCode Benchmark

A scientist-curated coding benchmark featuring 288 test set subproblems from 80 laboratory problems across 16 scientific disciplines.

Humanity's Last Exam BenchmarkHumanity's Last Exam Benchmark

A frontier-level benchmark with 2,500 expert-vetted questions across mathematics, sciences, and humanities, designed to be the final closed-ended academic evaluation.

CritPt BenchmarkCritPt Benchmark

A benchmark designed to test LLMs on research-level physics reasoning tasks, featuring 71 composite research challenges.

Terminal-Bench Hard BenchmarkTerminal-Bench Hard Benchmark

An agentic benchmark evaluating AI capabilities in terminal environments through software engineering, system administration, and data processing tasks.

𝜏²-Bench Telecom Benchmark𝜏²-Bench Telecom Benchmark

A dual-control conversational AI benchmark simulating technical support scenarios where both agent and user must coordinate actions to resolve telecom service issues.

MMLU-Pro BenchmarkMMLU-Pro Benchmark

An enhanced version of MMLU with 12,000 graduate-level questions across 14 subject areas, featuring ten answer options and deeper reasoning requirements.

LiveCodeBench BenchmarkLiveCodeBench Benchmark

A contamination-free coding benchmark that continuously harvests fresh competitive programming problems from LeetCode, AtCoder, and CodeForces, evaluating code generation, self-repair, and execution.

MATH-500 BenchmarkMATH-500 Benchmark

A 500-problem subset from the MATH dataset, featuring competition-level mathematics across six domains including algebra, geometry, and number theory.

AIME 2025 BenchmarkAIME 2025 Benchmark

All 30 problems from the 2025 American Invitational Mathematics Examination, testing olympiad-level mathematical reasoning with integer answers from 000-999.

Global-MMLU-Lite BenchmarkGlobal-MMLU-Lite Benchmark

A lightweight, multilingual version of MMLU, designed to evaluate knowledge and reasoning skills across a diverse range of languages and cultural contexts.