Easy Problems That LLMs Get Wrong

This MicroEval evaluates LLM responses to simple logic-based questions that LLMs commonly get wrong. The problems used in this MicroEval are from the ArXiv paper of the same name: https://arxiv.org/abs/2405.19616. It is by Sean Williams and James Huckle, so props to them for developing this experiment all the way back in 2024.