Follow us on Twitter or LinkedIn to stay up to date with future analysis

Artificial Analysis

Artificial Analysis
Hardware
AI Trends
MicroEvals
Articles

Footer

Key Links

Compare Language Models
Language Models Leaderboard
Language Model API Leaderboard
Image Arena
Video Arena
Speech Arena

Artificial Analysis

FAQ
Contact & Data access
Terms of Use
Privacy Policy
[email protected]

Subscribe to our newsletter

Email address

Twitter LinkedIn

StrawberryEval | MicroEval | Artificial Analysis

StrawberryEval

Create MicroEval

Header image for StrawberryEval

StrawberryEval

Easy questions that LLMs still trip up on

Prompts (4)

Models (14)

Gemini 2.5 Pro
DeepSeek R1 0528 (May '25)
GPT-4.1
Qwen3 235B A22B
Gemma 3 27B Instruct
Qwen3 8B
DeepSeek V3 0324 (Mar '25)
Claude 4 Sonnet
o3
Kimi K2
Grok 4
Llama 4 Maverick
Llama 4 Scout
Gemma 3n E4B Instruct Preview (May '25)

Prompt

How many rs are there in the word strawberry?

Answer guidance

The answer is still 3!

Drag to resize

DeepSeek R1 0528 (May '25)

Drag to resize

Drag to resize

Qwen3 235B A22B

Drag to resize

Gemma 3 27B Instruct

Drag to resize

Drag to resize

DeepSeek V3 0324 (Mar '25)

Drag to resize

Claude 4 Sonnet

Drag to resize

Drag to resize

Drag to resize

Drag to resize

Llama 4 Maverick

Drag to resize

Drag to resize

Gemma 3n E4B Instruct Preview (May '25)

Drag to resize