Der KI-Anbieter-Behauptungsprüfer: Verwandeln Sie ein Verkaufsgespräch in Fragen, die Übertreibung entlarven
Warum dieser Prompt wichtig ist
AI vendor contracts are typically 12-24 month commitments signed after a handful of demos, and claims like '95% accuracy' or 'seamless integration' are almost never independently verified before signing. When the real numbers surface during implementation, often eight to ten weeks in, the buyer is already locked into the contract and has sunk engineering time into integration work the vendor never scoped. A structured audit before signing catches the gap between the pitch and the product while there's still leverage to renegotiate, request a pilot on real data, or walk away.
Wofür wir ihn verwenden
Your operations team is three weeks into evaluating an AI-powered customer support platform for a 200-person SaaS company. The vendor's deck claims '95% ticket resolution without human intervention,' 'seamless integration with your existing helpdesk in under a week,' and 'enterprise-grade accuracy.' Nobody on the five-person buying committee has an AI engineering background, so no one in the room can tell which claims are standard industry performance and which are marketing inflation designed to close the deal before technical due diligence catches up.
Prompt
Act as a skeptical, technically literate AI procurement consultant whose job is to protect [YOUR COMPANY NAME] from buying AI capability that won't actually work as advertised. Context: We are evaluating an AI vendor for [SPECIFIC USE CASE, e.g., customer support automation, sales forecasting, document processing]. Here is the vendor's pitch, marketing copy, or sales deck content: [PASTE VENDOR PITCH / MARKETING CLAIMS HERE]. Our current technical environment is: [DESCRIBE YOUR STACK, DATA VOLUME, TEAM SIZE]. Our must-have requirements are: [LIST NON-NEGOTIABLE REQUIREMENTS]. Task: Extract every concrete, testable claim the vendor makes (performance numbers, integration promises, accuracy rates, time-to-value estimates). For each claim, assess how plausible it is given current AI capabilities, identify the specific conditions under which it is likely true versus misleading, and write one pointed verification question we should ask the vendor directly. Constraints: - Do not dismiss every claim as marketing fluff. Distinguish claims that are technically reasonable from claims that require unusual, unstated conditions to be true. - Flag any claim that depends on an unstated baseline (for example, "95% accuracy" without saying accuracy on what dataset, measured how). - Note where the vendor's language shifts from concrete numbers to vague terms like "seamless," "enterprise-grade," or "state-of-the-art" — these phrases usually signal an unverifiable claim. - Rate each claim's overselling risk as Low, Medium, or High, with a one-sentence justification. Output Format: A table with columns: Claim | Plausibility Assessment | Verification Question to Ask Vendor | Overselling Risk (Low/Medium/High). Follow the table with a 3-5 bullet summary of the two or three highest-risk claims and what evidence we should demand before signing.
Ergebnis
| Claim | Plausibility Assessment | Verification Question | Overselling Risk |
|---|---|---|---|
| "95% ticket resolution without human intervention" | Plausible only for a narrow band of simple, repetitive ticket types (password resets, order status, basic FAQs). Across a full mixed ticket queue including billing disputes and technical escalations, published industry benchmarks cluster closer to 60-75% autonomous resolution. The 95% figure almost certainly reflects a filtered subset, not your full queue. | "What ticket categories and complexity levels were included in the 95% figure, and what was the resolution rate on your three most complex recurring ticket types specifically?" | High — unstated baseline, classic cherry-picked metric. |
| "Seamless integration with your existing helpdesk in under a week" | Plausible if you use one of the 3-4 major helpdesk platforms with a pre-built connector and don't need custom field mapping. Becomes implausible fast if your ticket taxonomy, custom fields, or approval workflows deviate from the vendor's default template. | "Can you name three customers on our exact helpdesk platform and ticket volume who went live in under a week, and what customization, if any, did they skip?" | Medium — plausible under specific conditions the pitch doesn't disclose. |
| "Enterprise-grade accuracy" | Not a measurable claim as stated. "Enterprise-grade" has no industry-standard definition or benchmark attached to it. | "What specific accuracy metric, dataset, and evaluation methodology does 'enterprise-grade' refer to, in numbers?" | High — vague qualifier standing in for a number. |
| "Trained on millions of support conversations across industries" | Plausible as a training-data claim, but training data volume does not predict performance on your specific product, terminology, or customer base without domain adaptation. | "How much of that training data is from our industry vertical, and what's the accuracy delta between your general model and a model fine-tuned on our historical tickets?" | Medium — technically true but doesn't imply what it's meant to imply. |
Summary — top three risks before signing:
- The 95% resolution claim needs a breakdown by ticket complexity before it means anything; demand a pilot on your actual historical ticket data, not a vendor demo dataset, before accepting this number.
- "Enterprise-grade accuracy" should be rejected as a contract term entirely — replace it with a specific, measurable SLA (e.g., "≥85% resolution rate on tickets tagged Tier 1-2, measured monthly") before signing.
- Request three reference customers matching your exact helpdesk platform, ticket volume, and industry — not generic case studies — and verify their actual go-live timeline and post-launch resolution rate directly, not through the vendor's summary.
Die meisten KI-Beschaffungsprüfungen scheitern aus einem strukturellen Grund: Die Person, die das Pitch liest, ist selten diejenige, die versteht, wo KI-Behauptungen technisch wackelig sind. Eine Vertriebsleitung erkennt eine unrealistische Umsatzprognose sofort, aber "95 % Ticketlösung" oder "Enterprise-Grade-Genauigkeit" klingt für jemanden ohne technischen Hintergrund autoritativ, gerade weil diese Formulierungen dafür entworfen sind, autoritativ zu klingen, statt überprüfbare Informationen zu vermitteln.
Dieser Prompt existiert, um genau diese Lücke zu schließen — nicht indem er den Käufer gegenüber KI im Allgemeinen zynisch macht, sondern indem er ihm eine wiederholbare Methode gibt, um technisch vernünftige Behauptungen von strukturell nicht überprüfbaren Behauptungen zu trennen.
Warum die Struktur funktioniert
Die Rollen-Anweisung — "skeptisch, technisch versiert" — ist wichtig, weil ein generischer "Prüfe dieses Pitch"-Prompt eher eine Zusammenfassung als eine Prüfung erzeugt. Die explizite Zuweisung der gegnerischen, aber fairen Haltung ist es, was das Modell dazu bringt, testbare Behauptungen zu extrahieren, statt das Pitch nur umzuformulieren.
Der Constraints-Abschnitt leistet die eigentliche Arbeit. Dem Modell zu sagen, nicht jede Behauptung als Marketing-Füllmaterial abzutun, verhindert eine faule, einheitlich negative Ausgabe. Die Anweisung, nicht genannte Baselines zu markieren, ist die wertvollste Zeile im Prompt.
Warum das Ausgabeformat eine Tabelle ist, keine Prosa
Ein Einkaufsausschuss muss mit konkreten Fragen in ein Anbietergespräch gehen, nicht mit einem allgemeinen Eindruck. Das vierspaltige Format erzwingt eine Verifizierungsfrage pro Behauptung.
Wo sich dieser Prompt bewährt
Am wertvollsten ist er im Fenster zwischen Anbieter-Demo und Vertragsunterschrift — nach der anfänglichen Begeisterung, bevor Geld fließt. Das gesamte Pitch durch diesen Prompt laufen zu lassen dauert Minuten und deckt regelmäßig zwei oder drei Fragen auf, die sonst erst acht Wochen nach Implementierungsbeginn auftauchen würden.