StrawberryEval
Easy questions that LLMs still trip up on
Prompt
How many rs are there in the word strawberry?
Answer guidance
The answer is still 3!
GoogleDrag to resize
GoogleDrag to resize
Drag to resize
DeepSeekDrag to resize
OpenAIDrag to resize
DeepSeekDrag to resize
AlibabaDrag to resize
AnthropicDrag to resize
OpenAIDrag to resize
KimiDrag to resize
SpaceXAIDrag to resize
MetaDrag to resize
MetaDrag to resize
Drag to resize