Stay connected with us on X, Discord, and LinkedIn to stay up to date with future analysis
All MicroEvals
Strawrbrerrry [sic] eval
Create MicroEval
Header image for Strawrbrerrry [sic] eval

Strawrbrerrry [sic] eval

Reasoning should include the ability to generalize to unfamiliar words instead of memorizing answers. Let's see if models can detect the number of 'r's in the word "strawrbrerrry."

Prompt

How many letters 'r' are in the word "strawrbrerrry?"

Answer guidance

We're hoping for 6

Drag to resize
Drag to resize
Drag to resize
Drag to resize
Drag to resize
Drag to resize
Drag to resize
Drag to resize
Drag to resize
Drag to resize
Drag to resize