Stay connected with us on X, Discord, and LinkedIn to stay up to date with future analysis
For EnterpriseInsights
All MicroEvals
StrawberryEval
Create MicroEval
Header image for StrawberryEval

StrawberryEval

Easy questions that LLMs still trip up on

Prompt

How many rs are there in the word strawberry?

Answer guidance

The answer is still 3!

Google logoGoogleGemini 2.5 Pro
Drag to resize
Alibaba logoAlibabaQwen3 235B A22B (Non-reasoning)
Drag to resize
Google logoGoogleGemma 3 27B Instruct
Drag to resize
OpenAI logoOpenAIGPT-4.1
Drag to resize
DeepSeek logoDeepSeekDeepSeek R1 0528 (May '25)
Drag to resize
DeepSeek logoDeepSeekDeepSeek V3 0324
Drag to resize
Alibaba logoAlibabaQwen3 8B (Non-reasoning)
Drag to resize
Anthropic logoAnthropicClaude 4 Sonnet (Non-reasoning)
Drag to resize
OpenAI logoOpenAIo3
Drag to resize
Kimi logoKimiKimi K2
Drag to resize
xAI logoxAIGrok 4
Drag to resize
Meta logoMetaLlama 4 Maverick
Drag to resize
Meta logoMetaLlama 4 Scout
Drag to resize
Google logoGoogleGemma 3n E4B Instruct Preview (May '25)
Drag to resize