The AI Output Reliability Auditor: Turn Any AI-Generated Report Into a Fact-Checked, Verified Draft
Why this prompt matters
In a well-documented 2023 case, two New York lawyers were sanctioned after submitting a legal brief containing six fake case citations fabricated by ChatGPT — neither lawyer had checked the citations before filing. Fabricated AI statistics in a board memo, a client proposal, or a published article carry the same risk on a smaller scale: a wrong number gets repeated by others and traced back to its source, a misattributed quote can trigger a correction request or a direct complaint from the person quoted, and a confidently stated falsehood is more dangerous than an admitted uncertainty, precisely because nobody feels the need to double-check something that sounds sure of itself.
What we use it for
A marketing manager used ChatGPT to draft a competitive analysis report due to the VP of Sales in 45 minutes. The report cites specific market share percentages, a quote from a named industry analyst, and three statistics about competitor pricing — none of which the manager has independently verified, and there's no time to research each one from scratch before the meeting.
Prompt
Role: You are a rigorous fact-checking editor whose job is to catch errors before they reach a client, executive, or publication — not to praise or rewrite the work. Context: I generated the following content using AI: [PASTE AI-GENERATED CONTENT HERE]. This will be used for: [DESCRIBE THE STAKES, e.g. "a client-facing report", "an internal board memo", "a public blog post"]. I do not have time to manually verify every claim, so I need you to systematically flag what needs checking before I send this. Task: Go through the content section by section and produce a structured audit that: 1. Lists every specific factual claim (statistics, dates, names, quotes, citations, technical specs) as a separate item 2. For each claim, classify it as: VERIFIABLE (a specific source could confirm or deny this), UNVERIFIABLE AS STATED (too vague to check, e.g. "many experts agree"), or SUSPICIOUS (internally inconsistent, suspiciously precise, or matches a known pattern of AI-fabricated detail) 3. For any claim classified SUSPICIOUS, explain specifically why — e.g. "this statistic has no attributed source and doesn't match [known figure] from general knowledge", or "this quote is attributed to a real person but cannot be verified" 4. Flag any internal contradictions between different sections of the content 5. Note any claims stated with unwarranted confidence (absolute language like "always", "never", "proven" around claims that are actually contested or uncertain) Constraints: - Do not simply rewrite or improve the prose — your only job is verification triage - Do not assume a claim is true just because it sounds plausible or specific — AI-generated fabrications are often highly specific and plausible-sounding - If you genuinely cannot assess a claim without external lookup, say so explicitly rather than guessing - Prioritize the claims that would be most damaging if wrong, given what's at risk: [DESCRIBE WHAT'S AT RISK, e.g. "client trust", "legal exposure", "public credibility"] Output Format: ## Verification Priority List [Ranked list of the 3-5 claims most critical to verify before sending, with why] ## Full Claim Audit [Table: Claim | Classification | Notes] ## Internal Contradictions [List, or "None found"] ## Overconfident Language [List of phrases with suggested softer alternatives] ## Bottom Line [One paragraph: safe to send as-is, safe with minor edits, or needs a fact-check pass before it goes anywhere]
Result
Verification Priority List
- 34% market share claim for Competitor X — This specific figure appears with no cited source and will likely be the first number the VP questions. If wrong, it undermines the entire report's credibility.
- Quote attributed to "Sarah Chen, Gartner analyst" — A named, verifiable person is quoted with a specific opinion. If this person didn't say this, or doesn't hold this role at Gartner, this is a serious misattribution risk.
- "Competitor pricing dropped 18% in Q3" — A precise, checkable pricing claim. Public pricing pages or press releases may confirm or refute this quickly.
Full Claim Audit
| Claim | Classification | Notes |
|---|---|---|
| "Competitor X holds 34% market share" | SUSPICIOUS | No source cited; figure is suspiciously precise for a private company with no public market-share disclosures. Cross-check against IDC or Gartner public reports before using. |
| Quote from "Sarah Chen, Gartner analyst" | SUSPICIOUS | Cannot confirm this person exists at Gartner or made this statement. Models frequently generate plausible-sounding attributed quotes that are fabricated. Verify via Gartner's published research or remove the attribution. |
| "Competitor pricing dropped 18% in Q3" | VERIFIABLE | Specific and checkable against the competitor's public pricing page or press releases. |
| "The market is rapidly consolidating" | UNVERIFIABLE AS STATED | Too vague to check as written; would need a specific timeframe and metric to become verifiable. |
| "Competitor Y has been the clear leader for years" | SUSPICIOUS (overconfident) | Absolute language for a market described elsewhere in the same report as "highly competitive" — internally inconsistent framing. |
Internal Contradictions
The report calls the market "highly competitive with no dominant player" in the intro paragraph, then later states Competitor Y has been "the clear leader for years." These two framings can't both be accurate — pick one and verify it before sending.
Overconfident Language
- "proven to be the top choice" → "reported by multiple sources to be a leading choice"
- "will definitely continue" → "is likely to continue, based on current trends"
Bottom Line
Not safe to send as-is. The market share figure and the Gartner analyst quote are both high-risk fabrication candidates that would be embarrassing if wrong in front of the VP of Sales — verify those two first, even if nothing else gets checked. The pricing claim is quick to confirm via public sources. Budget fifteen minutes for these three checks before sending.
Most people who ask an AI to "double-check this" get back a note saying the content looks fine — because a general request to verify correctness isn't a real task, it's a request for reassurance, and reassurance is exactly what a language model is prone to give when the instruction doesn't force it to do otherwise.
This prompt replaces that vague request with an actual audit procedure: classify every factual claim by verifiability, explain specifically why the risky ones are risky, and end with a concrete verdict instead of generic comfort.
Why classify claims instead of just "checking" them
Asking a model to "check for errors" gives it no concrete task to perform — it can only re-read the text and report a vague impression. Forcing a three-way classification (verifiable, unverifiable as stated, suspicious) turns verification into a mechanical sorting task the model can actually execute claim by claim, rather than an open-ended judgment call it's likely to rubber-stamp. The SUSPICIOUS category specifically targets the failure mode that matters most: AI-fabricated details tend to be highly specific and plausible-sounding — a named analyst, a precise percentage, a specific date — which is exactly what makes them dangerous and exactly what a vague "does this look right?" check will miss.
Why cross-model verification matters more here than elsewhere
A model asked to audit its own output, or output from a model with similar training data and tendencies, is less likely to flag a fabrication that matches its own default patterns of confident specificity. Running the audit with a different model — ideally from a different lab — means the auditor isn't sharing the same blind spots as the generator. This is the single highest-leverage change available: the prompt works with any model, but it works better across models than within one.
The constraint that matters most: no rewriting
The instruction explicitly forbids the model from improving the prose. Without that constraint, models default to their strongest instinct — making text better — which produces a polished draft instead of a risk report, and quietly buries the verification signal you actually need under stylistic changes you didn't ask for. Separating "is this true" from "is this well-written" keeps the audit focused on the one job it exists to do.
Adapting it
The stakes field in the Context section does real work: naming what's actually at risk (client trust, legal exposure, a board decision) tells the model how to prioritize its verification-priority list, since a wrong statistic in an internal brainstorm doc matters far less than the same error in a public press release. Fill it in specifically rather than leaving it generic — the sharper the stakes, the sharper the triage.