The Metric Gaming Detector: Find Out If Your KPI Improved or Just Got Easier to Hit

Why this prompt matters
KPI gaming that goes undetected for one more quarter doesn't just produce a wrong number — it gets copied. The same incentive structure, the same shortcut, rolled out to two more teams, now sits underneath headcount plans, comp decisions, and board-level reporting. The 2016 Wells Fargo cross-selling scandal is the textbook version of this: a sales metric kept improving because of how it was being gamed, not because of a change in actual customer value, and the gap widened for years before anyone checked. Auditing a single metric takes thirty minutes; unwinding a comp plan and a reporting narrative built on a fake improvement takes months and a credibility hit no apology fixes.
What we use it for
Your support team's average ticket resolution time dropped from 18 hours to 6 hours over one quarter, and leadership wants to roll the same resolution-time bonus out to two more teams before anyone has confirmed the number reflects real improvement rather than a reporting change.
Prompt
Act as a skeptical analytics auditor who has spent a decade finding where teams quietly game performance metrics without technically lying. Context: - Metric name: [METRIC NAME] - Baseline value: [BASELINE VALUE] - Current value: [CURRENT VALUE] - Time period: [TIME PERIOD] - How the metric is currently calculated: [PASTE METRIC DEFINITION / CALCULATION METHOD] - What changed in how the team operates, what changed in tooling or policy, and any anecdotes from the team during this period: [PASTE PROCESS, TOOLING, OR POLICY CHANGES] Task: Determine whether this improvement reflects genuine underlying performance, or whether it is partially or fully explained by a change in measurement, scope, incentive gaming, or selection effects. For each explanation you generate, rate its plausibility specifically against the evidence provided — not generic hedging — and state exactly what evidence would need to exist to confirm or rule out each one. Constraints: - Assume by default that any improvement greater than 20% in a short window is suspicious until proven otherwise. - Do not suggest punishing the team; the goal is to find the truth, not assign blame. - Flag explicitly if the metric's definition has known gaming vectors (for example: survivorship bias, denominator manipulation, exclusion creep, or auto-closure). - If the information provided is insufficient to make a judgment, say precisely what additional data you would need rather than guessing. Output Format: 1. VERDICT — one paragraph stating your overall confidence that the improvement is real (Likely Real / Mixed / Likely Gamed / Cannot Determine) with a one-line justification. 2. EXPLANATIONS TABLE — a markdown table with columns: Explanation | Plausibility (Low/Medium/High) | Supporting Evidence | What Would Confirm/Rule This Out. Include at least 4 distinct explanations: at least one genuine-improvement explanation, and at least two gaming or measurement-artifact explanations. 3. NEXT AUDIT STEP — the single most informative question to ask the team, or dataset to pull, that would most efficiently resolve the remaining uncertainty.
Result
VERDICT: Likely Mixed. The tiered triage system launched in week 3 probably contributed a real improvement in resolution time for simpler tickets, but the resolution-time-linked bonus that started in week 1 of the same quarter, combined with informal reports of rising reopens, is a classic signature of incentive-driven premature closure — and the data provided cannot rule that out.
| Explanation | Plausibility | Supporting Evidence | What Would Confirm/Rule This Out |
|---|---|---|---|
| Genuine process improvement — tiered triage routes simple tickets to faster resolution paths | Medium | Triage launched week 3; the drop in average resolution time continued through week 13 | Break resolution time out by ticket complexity tier. Real improvement should show up specifically on the tickets triage was designed to speed up, not uniformly across all ticket types. |
| Incentive-driven premature closure — agents close tickets before the issue is fixed to hit the bonus threshold | High | Bonus introduced the same week the measurement window starts; agents mentioned in Slack that “reopens are way up” | Pull the 7-day reopen rate for the same period. If reopens have roughly doubled, a meaningful share of “resolved” tickets were closed prematurely. |
| Denominator change — new routing logic silently auto-closes or merges some tickets, removing them from the human-handled average | Medium | Tiered systems commonly introduce auto-resolution or duplicate-merge rules as part of the same rollout | Pull raw ticket volume by resolution method (human-closed vs. auto-closed) before and after triage launched. |
| Backlog-clearing selection effect — an old backlog of slow tickets was cleared early in the quarter, mechanically lowering the rolling average | Low | No backlog-clearing initiative was mentioned in the context provided | Check resolution time for tickets opened only in weeks 1-2, isolated from the trailing average, to see if new-ticket handling speed actually changed. |
NEXT AUDIT STEP: Pull the 7-day reopen rate for this team for the four weeks before and after the triage-and-bonus launch. If reopens haven't moved, premature closure is unlikely and the 6-hour figure can largely be trusted. If reopens have roughly doubled or more, treat the number as not-yet-real and hold off rolling the bonus structure out to the other two teams until triage and compensation are decoupled.
A metric moving in the right direction gets celebrated before anyone checks whether it means what everyone assumes it means. This prompt is built around a specific failure mode: large, fast improvements in a KPI are more often explained by a change in measurement, a change in incentives, or a change in what gets counted than by a genuine change in underlying performance — and teams rarely check which one happened before rolling the “win” out further.
The default-suspicious framing in the Constraints section is deliberate. Most audit prompts quietly flatter whoever is asking, producing a validation exercise instead of an audit. Forcing the model to treat any improvement above 20% in a short window as suspicious until proven otherwise removes that bias and makes the output useful before a decision gets made, not just after something has already gone wrong.
The Explanations Table requires at least one genuine-improvement explanation and at least two gaming or measurement-artifact explanations. That ratio exists because a one-sided audit — every possible way the number could be fake, none of the ways it could be real — is just as useless as an audit that only confirms what leadership wants to hear. Both failure modes lead to the same bad decision.
Pairing every explanation with “what would confirm or rule this out” is what makes the output actionable instead of a list of suspicions. A plausible-sounding theory that can't be checked against real data is speculation; tying it to a specific pull — a reopen rate, a denominator breakdown, a cohort comparison — turns the audit into a to-do list instead of a worry list.
The single Next Audit Step at the end exists because teams facing an ambiguous metric tend to either do nothing, deferring the question indefinitely, or audit everything, stalling the rollout for weeks. Naming the one highest-leverage check forces a decision: get that one piece of evidence, then move.