Follow us on Twitter or LinkedIn to stay up to date with future analysis

Artificial Analysis Insights Login

Artificial Analysis
Hardware
AI Trends
MicroEvals
Beta
Articles

Footer

Key Links

Compare Language Models
Language Models Leaderboard
Language Model API Leaderboard
Image Arena
Video Arena
Speech Arena

Artificial Analysis

FAQ
Contact & Data access
Terms of Use
Privacy Policy
[email protected]

Subscribe to our newsletter

Email address

Twitter LinkedIn

StrawberryEval | MicroEval | Artificial Analysis

StrawberryEval

Create MicroEval

Header image for StrawberryEval

StrawberryEval

Easy questions that LLMs still trip up on

Prompts (4)

Models (10)

Gemini 2.5 Pro
GPT-4.1
Qwen3 235B A22B
DeepSeek R1 0528 (May '25)
Gemma 3 27B Instruct
DeepSeek V3 0324 (Mar '25)
Qwen3 8B
Claude 4 Sonnet
Llama 4 Maverick
Llama 4 Scout

Prompt

How many rs are there in the word strawberry?

Answer guidance

The answer is still 3!

Drag to resize

Drag to resize

Qwen3 235B A22B

Drag to resize

DeepSeek R1 0528 (May '25)

Drag to resize

Gemma 3 27B Instruct

Drag to resize

DeepSeek V3 0324 (Mar '25)

Drag to resize

Drag to resize

Claude 4 Sonnet

Drag to resize

Llama 4 Maverick

Drag to resize

Drag to resize