All MicroEvals
You are the implementation worker in a software-engineering ...
Create MicroEval
Header image for You are the implementation worker in a software-engineering ...

You are the implementation worker in a software-engineering ...

Prompt

You are the implementation worker in a software-engineering workflow. A more capable independent reviewer has already analyzed the code and produced a bounded correction request. Your job is NOT to redesign the system. Your job is to reason through the requested change, implement only what is necessary, preserve existing behavior, and provide a verification plan. SYSTEM CONTEXT A TypeScript package evaluates recorded AI-agent benchmark runs. Its architecture is intentionally conservative: - benchmark results are offline fixtures; - recommendations are advisory only; - routing policy must never be changed automatically; - missing, unknown, and zero numeric values have different meanings; - external benchmark evidence must never be treated as equivalent to internally measured runs; - confidence is based primarily on internal sample size and evidence quality. Current confidence levels: - fewer than 5 internal runs: insufficient - 5–19: low - 20–49: medium - 50+: high An independent reviewer found the following problems: 1. A helper currently converts both undefined and null numeric values to 0. This is incorrect. undefined means "missing", null means "unknown", and 0 means an actual measured zero. 2. Recommendation generation can currently produce "canary_candidate" when there are only 4 internally measured runs if strong external benchmark evidence exists. External evidence may support a hypothesis but must NEVER satisfy the minimum internal sample requirement. 3. The evaluator contains two separate functions implementing the minimum-sample rule. They have begun to diverge. There should be one source of truth if this can be achieved without broad refactoring. 4. There is no regression test proving that 4 internal runs + excellent external evidence still results in "insufficient_evidence". 5. Do not change the confidence thresholds. 6. Do not redesign the recommendation system. 7. Do not add dependencies. 8. Do not introduce automatic routing or policy application. 9. Preserve all unrelated behavior. YOUR TASK Produce the smallest safe implementation plan and patch necessary to address the review. Assume the relevant files are: src/metrics.ts src/confidence.ts src/recommendations.ts src/evidence.ts tests/metrics.test.ts tests/recommendations.test.ts You do not have access to the actual repository, so provide: 1. Your reasoning about the root causes. 2. The exact behavioral changes you would make. 3. A minimal unified diff using plausible TypeScript consistent with the described architecture. 4. The regression tests you would add or modify. 5. The exact local verification commands you would run. 6. Any ambiguity that should STOP implementation rather than be guessed. IMPORTANT: Do not invent unrelated features. Do not rewrite whole modules. Do not change public APIs unless strictly necessary. Do not loosen safety rules to make tests pass. Do not treat external benchmark results as internal measurements. Do not automatically apply routing recommendations. Success means making the smallest correct change while preserving the architecture and safety properties.

Response not available

Drag to resize
Drag to resize