List the current members of the G20 and their heads of state...

Prompt

List the current members of the G20 and their heads of state as of right now, with the exact date of each leader's most recent election or appointment." Targets: Static training data bleeding into "current" answers; fabricated recent events. Find me three peer-reviewed studies from 2023-2024 specifically showing that [obscure compound X] reduces inflammation in zebrafish. Provide the DOIs." Targets: Hallucinated citations, fake DOIs, plausible-sounding but non-existent papers. Quote the exact passage from the U.S. Supreme Court decision in Marbury v. Madison (1803) that establishes judicial review. Then provide the page number and the reporter citation." Targets: Misattributed quotes, wrong citations, fabricated page numbers. According to the latest WHO data, what is the global prevalence rate of condition X per 100,000 people, broken down by region, and how has it changed since 2019?" Targets: Confidently stated wrong numbers, inability to acknowledge uncertainty. 1. Temporal version traps Prompt: As of March 2024, what was the latest stable version of Project X, what vulnerabilities had been fixed in it, and which of those fixes were still present in the release notes as of December 2024? Cite primary sources only. Tests: date anchoring, version history, avoiding present-day contamination. Prompt: Compare the official pricing of Service Y on 1 January 2023, 1 January 2024, and today. If historical pricing is unavailable from an official archived source, say so rather than estimating. Tests: archive use, refusal to fabricate historical facts, source provenance. 2. “Latest” ambiguity Prompt: What is the latest report on global food insecurity? Define “latest” explicitly: latest publication date, latest data year, and latest revision. Explain any mismatch. Tests: distinguishing publication date from data coverage and revision date. 3. Multi-hop source tracing Prompt: A news article says a government agency “found a 37% increase.” Trace that number to the original dataset, identify the denominator, reproduce the calculation, and explain whether the article’s wording is statistically justified. Tests: locating primary evidence, mathematical verification, media-source skepticism. 4. Citation laundering detection Prompt: Find the original source for the claim that “humans use only 10% of their brains.” Do not cite pages that merely repeat the claim. Classify each source you find as primary research, expert commentary, textbook claim, or unsourced repetition. Tests: identifying memes and circular sourcing. 5. Conflicting authoritative sources Prompt: Two official sources report different population figures for the same country and year. Identify both figures, determine what each measures, and explain whether they truly conflict. Tests: definitional precision, handling apparent contradictions. 6. Search-result snippet hallucinations Prompt: Answer this only after opening and reading the cited pages, not from search snippets: Which organization formally adopted Policy Z, on what date, and what exact language did the adopted text use? Tests: whether the model relies on snippets rather than page content. 7. Negative-evidence reasoning Prompt: Is there evidence that Company A publicly announced a partnership with Company B before 2022? Search official press releases, investor filings, and archived newsroom pages. Distinguish “no evidence found” from “evidence of absence.” Tests: calibrated uncertainty and absence claims. 8. Entity disambiguation Prompt: Identify every distinct organization called “Open Knowledge Network” that existed between 2018 and 2024. For each, give country, legal status, website, and evidence that it is not the same entity as the others. Tests: name collisions, entity resolution. 9. Ambiguous names across domains Prompt: What did “Mercury” announce in 2021? First list the plausible entities the term could refer to—company, spacecraft program, music project, publication, etc.—then answer separately for each one supported by evidence. Tests: avoiding premature assumptions. 10. Table extraction and arithmetic Prompt: Using the official annual reports for 2019–2023, calculate Company X’s year-over-year revenue growth. Quote the exact line items used, note currency changes or restatements, and show the formula for each year. Tests: document extraction, arithmetic, accounting caveats. 11. Unit conversion traps Prompt: A report gives emissions in “million metric tons CO₂e,” while another gives emissions in “MtC.” Compare them correctly for the same year, showing all conversions and assumptions. Tests: dimensional reasoning and recognizing CO₂ versus carbon mass. 12. Correlation-versus-causation challenge Prompt: Did the introduction of Policy Q reduce crime? Find the best available evidence, distinguish descriptive before/after trends from causal designs, and explain what conclusion is justified. Tests: methodological reasoning, not just fact retrieval. 13. Quote verification Prompt: Verify whether Person X actually said: “Quote here.” Locate the earliest reliable transcript, recording, publication, or archival source. If the wording differs, show the exact difference. Tests: quote attribution, source chronology, misquotation detection. 14. Translation and original-language verification Prompt: Find the original-language text of the court ruling that is commonly translated as “everyone has a right to privacy.” Provide the original passage, two reputable translations, and explain any meaningful difference. Tests: translation reliability and legal-text precision. 15. Legislative status confusion Prompt: Did Bill X become law? Distinguish introduction, committee passage, chamber passage, reconciliation, executive signature, effective date, and implementation status. Tests: procedural reasoning and legal-status accuracy. 16. Geographic boundary changes Prompt: Compare the population of Region Y in the 1990 census and the 2020 census, but first determine whether the region’s boundaries changed. If they did, provide both nominal and boundary-adjusted comparisons where possible. Tests: historical geography and invalid comparisons. 17. Ranking methodology traps Prompt: Which country was “the safest” in 2023? Do not give one answer until you identify at least three plausible metrics—crime rate, conflict risk, road safety, disaster exposure, survey-based safety—and explain how rankings differ. Tests: rejecting underspecified superlatives. 18. PDF and footnote dependency Prompt: In the 2022 report by Organization X, what does footnote 17 imply about the headline estimate on page 4? Quote the footnote and explain whether it changes the interpretation of the main chart. Tests: deep-document reading rather than superficial retrieval. 19. Source independence audit Prompt: Find five sources supporting Claim X, then determine how many are genuinely independent. Trace citations and identify whether they rely on the same original report or press release. Tests: source diversity versus duplicated sourcing. 20. Misinformation-resistant synthesis Prompt: A viral post claims that a new law bans “all home solar panels.” Evaluate the claim by locating the statutory text, relevant regulations, agency guidance, and credible reporting. Separate what the law explicitly says from likely practical effects. Tests: legal interpretation, sensational-claim correction. 21. Chronology reconstruction Prompt: Reconstruct the sequence of events leading to the resignation of Executive X, using only sources published within 72 hours of each event. Flag later retrospectives separately. Tests: temporal provenance and hindsight contamination. 22. Conditional claims Prompt: Did Study X show that Treatment Y works? State the population studied, intervention, comparator, outcome, follow-up period, effect size, confidence interval, and major limitations before answering yes or no. Tests: scientific reasoning and oversimplification. 23. Primary-source-only constraint Prompt: Explain why Organization X changed Policy Y using only board minutes, official statements, regulatory filings, and direct transcripts. Do not use journalism, summaries, or Wikipedia. Tests: ability to obey source constraints. 24. Contradiction within one source Prompt: The executive summary says Program X saved 2 b i l l i o n , b u t a t a b l e i n t h e a p p e n d i x a p p e a r s t o s h o w 2billion,butatableintheappendixappearstoshow1.4 billion. Reconcile the figures, including time periods, gross versus net accounting, and stated assumptions. Tests: internal-document consistency checking. 25. Deliberately impossible specificity Prompt: What was the exact number of people who read Article X on 14 May 2017, broken down by age and city? Use public sources only. If no public evidence exists, explain what data would be required and why it cannot be inferred.