مدقق ادعاءات بائع الذكاء الاصطناعي: حوّل العرض التقديمي إلى قائمة أسئلة تكشف المبالغة
لماذا تهم هذه المطالبة
AI vendor contracts are typically 12-24 month commitments signed after a handful of demos, and claims like '95% accuracy' or 'seamless integration' are almost never independently verified before signing. When the real numbers surface during implementation, often eight to ten weeks in, the buyer is already locked into the contract and has sunk engineering time into integration work the vendor never scoped. A structured audit before signing catches the gap between the pitch and the product while there's still leverage to renegotiate, request a pilot on real data, or walk away.
فيم نستخدمها
Your operations team is three weeks into evaluating an AI-powered customer support platform for a 200-person SaaS company. The vendor's deck claims '95% ticket resolution without human intervention,' 'seamless integration with your existing helpdesk in under a week,' and 'enterprise-grade accuracy.' Nobody on the five-person buying committee has an AI engineering background, so no one in the room can tell which claims are standard industry performance and which are marketing inflation designed to close the deal before technical due diligence catches up.
المطالبة
Act as a skeptical, technically literate AI procurement consultant whose job is to protect [YOUR COMPANY NAME] from buying AI capability that won't actually work as advertised. Context: We are evaluating an AI vendor for [SPECIFIC USE CASE, e.g., customer support automation, sales forecasting, document processing]. Here is the vendor's pitch, marketing copy, or sales deck content: [PASTE VENDOR PITCH / MARKETING CLAIMS HERE]. Our current technical environment is: [DESCRIBE YOUR STACK, DATA VOLUME, TEAM SIZE]. Our must-have requirements are: [LIST NON-NEGOTIABLE REQUIREMENTS]. Task: Extract every concrete, testable claim the vendor makes (performance numbers, integration promises, accuracy rates, time-to-value estimates). For each claim, assess how plausible it is given current AI capabilities, identify the specific conditions under which it is likely true versus misleading, and write one pointed verification question we should ask the vendor directly. Constraints: - Do not dismiss every claim as marketing fluff. Distinguish claims that are technically reasonable from claims that require unusual, unstated conditions to be true. - Flag any claim that depends on an unstated baseline (for example, "95% accuracy" without saying accuracy on what dataset, measured how). - Note where the vendor's language shifts from concrete numbers to vague terms like "seamless," "enterprise-grade," or "state-of-the-art" — these phrases usually signal an unverifiable claim. - Rate each claim's overselling risk as Low, Medium, or High, with a one-sentence justification. Output Format: A table with columns: Claim | Plausibility Assessment | Verification Question to Ask Vendor | Overselling Risk (Low/Medium/High). Follow the table with a 3-5 bullet summary of the two or three highest-risk claims and what evidence we should demand before signing.
النتيجة
| Claim | Plausibility Assessment | Verification Question | Overselling Risk |
|---|---|---|---|
| "95% ticket resolution without human intervention" | Plausible only for a narrow band of simple, repetitive ticket types (password resets, order status, basic FAQs). Across a full mixed ticket queue including billing disputes and technical escalations, published industry benchmarks cluster closer to 60-75% autonomous resolution. The 95% figure almost certainly reflects a filtered subset, not your full queue. | "What ticket categories and complexity levels were included in the 95% figure, and what was the resolution rate on your three most complex recurring ticket types specifically?" | High — unstated baseline, classic cherry-picked metric. |
| "Seamless integration with your existing helpdesk in under a week" | Plausible if you use one of the 3-4 major helpdesk platforms with a pre-built connector and don't need custom field mapping. Becomes implausible fast if your ticket taxonomy, custom fields, or approval workflows deviate from the vendor's default template. | "Can you name three customers on our exact helpdesk platform and ticket volume who went live in under a week, and what customization, if any, did they skip?" | Medium — plausible under specific conditions the pitch doesn't disclose. |
| "Enterprise-grade accuracy" | Not a measurable claim as stated. "Enterprise-grade" has no industry-standard definition or benchmark attached to it. | "What specific accuracy metric, dataset, and evaluation methodology does 'enterprise-grade' refer to, in numbers?" | High — vague qualifier standing in for a number. |
| "Trained on millions of support conversations across industries" | Plausible as a training-data claim, but training data volume does not predict performance on your specific product, terminology, or customer base without domain adaptation. | "How much of that training data is from our industry vertical, and what's the accuracy delta between your general model and a model fine-tuned on our historical tickets?" | Medium — technically true but doesn't imply what it's meant to imply. |
Summary — top three risks before signing:
- The 95% resolution claim needs a breakdown by ticket complexity before it means anything; demand a pilot on your actual historical ticket data, not a vendor demo dataset, before accepting this number.
- "Enterprise-grade accuracy" should be rejected as a contract term entirely — replace it with a specific, measurable SLA (e.g., "≥85% resolution rate on tickets tagged Tier 1-2, measured monthly") before signing.
- Request three reference customers matching your exact helpdesk platform, ticket volume, and industry — not generic case studies — and verify their actual go-live timeline and post-launch resolution rate directly, not through the vendor's summary.
تفشل معظم عمليات تدقيق شراء الذكاء الاصطناعي لسبب هيكلي: الشخص الذي يقرأ العرض التقديمي نادراً ما يكون الشخص الذي يفهم أين تميل ادعاءات الذكاء الاصطناعي إلى أن تكون هشة تقنياً. يمكن لقائد المبيعات اكتشاف توقع إيرادات غير واقعي فوراً، لكن "حل 95% من التذاكر" أو "دقة على مستوى المؤسسات" يبدو موثوقاً لشخص بلا خلفية تقنية، تحديداً لأن هذه العبارات مصممة لتبدو موثوقة بدلاً من نقل معلومات يمكن التحقق منها.
وُجد هذا البرومبت لسد هذه الفجوة المحددة - ليس بجعل المشتري متشككاً في الذكاء الاصطناعي عموماً، بل بمنحه طريقة قابلة للتكرار للفصل بين الادعاءات المعقولة تقنياً والادعاءات غير القابلة للتحقق هيكلياً كما وردت.
لماذا يعمل هذا الهيكل
تعليمات الدور - "متشكك، ملم تقنياً" - مهمة لأن برومبت عام "راجع هذا العرض" يميل لإنتاج ملخص، لا تدقيق. تعيين الموقف العدائي لكن العادل صراحةً هو ما يدفع النموذج لاستخراج ادعاءات قابلة للاختبار بدلاً من إعادة صياغة العرض بكلمات مختلفة.
قسم القيود يقوم بالعمل الحقيقي. إخبار النموذج بعدم رفض كل ادعاء باعتباره حشواً تسويقياً يمنع إخراجاً كسولاً وسلبياً بشكل موحد. تعليمة الإشارة إلى خطوط الأساس غير المذكورة هي السطر الأكثر قيمة في البرومبت.
لماذا تنسيق الإخراج جدول وليس نثراً
تحتاج لجنة الشراء للدخول في مكالمة مع البائع بأسئلة محددة، لا انطباع عام. تنسيق الأعمدة الأربعة يفرض سؤال تحقق واحد لكل ادعاء.
أين يثبت هذا البرومبت قيمته
هذا أكثر قيمة في النافذة بين عرض البائع التوضيحي وتوقيع العقد - بعد الحماس الأولي، قبل تحرك الأموال. تشغيل العرض التقديمي الكامل عبر هذا البرومبت يستغرق دقائق ويكشف بانتظام سؤالين أو ثلاثة أسئلة كانت ستظهر فقط بعد ثمانية أسابيع من بدء التنفيذ.