All MicroEvals
I'm asking several AI models the same question to compare th...
Create MicroEval

I'm asking several AI models the same question to compare th...

Prompt

I'm asking several AI models the same question to compare their ideas. Please answer independently and in the format requested at the end. ## The project A free study app built for one person preparing for the Auditing and Attestation (AUD) section of the US CPA exam. It was built with an AI model (Claude) as the main developer. The learner: - She has an MS in Accounting and a couple of years of accounting work, and has not started CPA study. - She studies 1-2 hours a day, mostly on an iPhone, sometimes on a laptop. - She prefers practice questions, short lessons and explanations over dense reading. - Her target sitting is January to June 2027, flexible. That date falls under the 2027 AUD blueprint: 4 hours, 78 multiple-choice questions in two testlets of 39, and 7 task-based simulations, weighted 50/50. - Area weights: I Ethics and responsibilities 15-25%, II Risk assessment 25-35%, III Further procedures and evidence 30-40%, IV Reporting 10-20%. - She has said she wants the app to be "as strong as Becker", and she wants mock exams. ## What it is today A single-file web app, opened in Safari on her phone. Progress is saved in the browser, with a manual backup/restore. Content is organized into topics set at one fictional client, a food distributor. Content (3 topics so far): 1. Assertions and audit evidence. 2. Substantive procedures: analytical procedures, confirmations, inventory. 3. Engagement terms, communication with governance, and audit documentation. Totals: about 90 reviewed multiple-choice questions. 16 of them are held back unseen for future mock exams. There are also 3 lessons, 3 short simulations, 10 recap cards and about 28 vetted YouTube links. Most of Areas I, II and IV is not covered yet. Features: - Learn sets. Each answer option has its own explanation, and every explanation quotes and cites the specific AICPA standard paragraph it rests on. - A spaced review queue, with items due on calendar days. - A timed set with no feedback until the end, and a delayed follow-up check the next day. - Short lessons (about 5 minutes): a dialogue between two recurring characters (an audit senior and a first-year associate), with static character portraits, two decision points per lesson, and read-aloud through the phone's built-in text-to-speech. She can open lessons in any order. - Mini simulations with exhibit tables, scored per response. - Recap cards, one per objective. - An "Ask Claude" chat that can explain what is on screen or answer general AUD questions. - First-try accuracy shown as a percentage, labelled as practice accuracy, not a prediction of an exam score. - Previous-question navigation, a Saved list, an All questions list, flags for items she thinks are wrong, and a Topics and Lessons list. How content is made and checked: - The AI drafts each item from the AICPA standards text, with a quote of 40 words or fewer from the paragraph it rests on. - A script verifies each quote against the source text. - A different model (ChatGPT) then answers each item blind, then checks the answer key, the quote, every wrong-option explanation and the full explanation. It classifies each issue as blocking or minor. - Blocking items are fixed and re-checked in a fresh chat. - A topic of about 46 items takes roughly 40 minutes of drafting and review time. - No human expert reviews the content. ## The current project rules (you may ignore any of them) 1. No new spending: only tools already paid for (Claude, ChatGPT, Google AI Pro) plus free tiers and free hosting. 2. Free official sources only; no licensed or purchased question banks or course content. 3. No human reviewer; accuracy is checked by AI cross-check only. 4. Built for one learner: no accounts, multi-user features or polish beyond her needs. 5. Phone-first design. 6. Build the smallest version that can be tested with her; do not add features just because they were suggested. 7. The developer AI does not make images, audio or video; those come from other tools, and narration uses the phone's text-to-speech. 8. Decisions are logged and not reopened without new information. 9. Practice scores are never presented as predictions of passing. 10. Accuracy matters more than volume or polish. ## What I'm asking Set the rules above aside. Tell me what you would change, add or remove to give her the best chance of passing AUD, and of studying effectively along the way. Think about what she should be able to do by early 2027. Be concrete. "Add gamification" is not useful; "a 10-question daily mixed review drawn from all topics, weighted toward her misses" is. Where you rely on learning research, name the principle (for example, retrieval practice or interleaving). Where you make claims about Becker, other review courses, the exam or the AICPA, mark them [assumed] unless you are certain. Do not invent statistics. ## Format (please follow it so I can compare answers) 1. Top 10 ideas, ranked by expected effect on her result. For each, give: - The idea, in one or two sentences. - Why it would help. - Rules it breaks, by number, or "none". - Cost in money: none, under $50, or more (say how much). - Effort to build: small, medium or large. - Risk to accuracy: low, medium or high, and why. - The smallest test that would show whether it works. 2. Quick wins: up to 5 things that break no rules. 3. Rules worth bending: which of the 10 rules you would change or drop, and why. Also which you would keep even without constraints. 4. What to stop or cut: anything in the current app or process that is not worth her time. 5. Biggest risk: the single thing most likely to stop this app from preparing her well, and what to do about it. 6. What you would need to know to give better advice. Keep the whole answer under 1,500 words.

Response not available

Drag to resize
Drag to resize
Drag to resize
Drag to resize
Drag to resize