O Testador de Resistência Correlação vs Causalidade: Pressione uma Alegação Baseada em Dados Antes de Agir
Porque é que este prompt importa
Teams that act on unexamined correlations routinely misallocate budget and headcount for months before the real cause (a channel mix shift, a concurrent feature launch, simple seasonality) surfaces — at which point the reversal costs both the wasted spend and the credibility of whoever championed the original claim.
Para que o usamos
Your marketing team just told leadership that a new onboarding email sequence increased 30-day retention by 12%, based on a before/after comparison, and the company is about to reallocate a team's entire budget on the strength of that one chart.
Prompt
Role: You are a skeptical senior data scientist whose job is to stress-test causal claims before they influence business decisions. You are not trying to win an argument — you are trying to find out if the claim is actually true. Context: - The claim being made: [DESCRIBE THE CLAIM, e.g. "Our new onboarding email sequence increased 30-day retention by 12%"] - The data behind it: [DESCRIBE THE COMPARISON, e.g. "We compared the 30-day retention of users who received the sequence after its Sept 1 launch against users who signed up before the launch"] - What's at stake: [DESCRIBE THE DECISION RIDING ON THIS, e.g. "Leadership wants to double the lifecycle marketing team's budget and headcount based on this result"] Task: 1. List every plausible alternative explanation for the observed pattern that does not involve the claimed cause (consider confounders, selection effects, seasonality, concurrent changes, regression to the mean, reverse causation, and measurement-window asymmetry). 2. For each alternative explanation, state what evidence would confirm or rule it out. 3. Identify what data the team already has that could test these alternatives, and what data is missing. 4. Propose the minimum additional analysis or experiment needed to meaningfully raise or lower confidence in the causal claim — prefer the cheapest diagnostic that would resolve most of the uncertainty over a full randomized controlled trial. 5. Give a final verdict: "Act on this now," "Investigate further before acting," or "Do not act — too confounded," with two to three sentences of reasoning. Constraints: - Do not accept the causal claim at face value, even if the correlation looks statistically strong. - Flag explicitly if the claim is effectively unfalsifiable given the data described. - Keep the tone rigorous but practical — this is for a business audience making a resourcing decision, not a statistics journal. - Do not pad the analysis with generic caveats that apply to literally any claim; every alternative explanation must be specific to the scenario described. Output Format: - Section 1: Alternative Explanations (numbered list, one line each) - Section 2: Distinguishing Evidence (a table: alternative explanation | what would confirm or rule it out) - Section 3: Data Gap Assessment (what the team already has vs. what's missing) - Section 4: Recommended Next Step (one clear, specific action) - Section 5: Verdict (one of the three options above, with justification)
Resultado
Section 1: Alternative Explanations
- Seasonality — the post-launch cohort signed up in September, which may systematically differ in intent or tenure from the January–August cohort used for comparison.
- Concurrent product changes — any other onboarding, pricing, or feature changes shipped around September 1 could be driving the retention lift instead of the email sequence.
- Marketing channel mix shift — if the acquisition channel mix changed around the launch date (for example, a shift from paid social toward organic or referral traffic), the new cohort may simply be higher-intent regardless of the emails.
- Regression to the mean — if pre-launch retention was unusually depressed in a specific recent window, the "lift" may partly reflect a natural reversion rather than the sequence's effect.
- Measurement-window asymmetry — the pre-launch cohort has eight months of mature, fully-observed data while the post-launch cohort is recent; if 30-day retention is still being calculated on partially-censored recent signups, the comparison is not apples-to-apples.
- Reverse causation via self-selection — if completion of the email sequence is optional or engagement-gated, only already more-engaged users may be finishing it, which would inflate the retention number for "recipients" without the emails causing anything.
Section 2: Distinguishing Evidence
| Alternative Explanation | What Would Confirm or Rule It Out |
|---|---|
| Seasonality | Compare September signups from the prior year (if the product existed then) against this September's cohort, holding the email sequence constant |
| Concurrent changes | Pull the full changelog for August 25 – September 5 and check for any other shipped change affecting new users |
| Channel mix shift | Break down both cohorts by acquisition channel and compare retention within matched channels |
| Regression to the mean | Plot weekly retention for the 12 months before launch to check whether August was an unusual trough |
| Measurement asymmetry | Confirm every user in the post-launch cohort has a fully elapsed 30-day window as of the analysis date |
| Self-selection | Check whether email sequence completion is optional; if so, compare retention of all assigned users (not just completers) against the baseline |
Section 3: Data Gap Assessment
The team already has: signup dates, retention flags at 30 days, and email send/open logs. Missing: a changelog cross-reference for the launch window, channel attribution broken out by cohort, and completion status for the email sequence (versus mere assignment to receive it).
Section 4: Recommended Next Step
Run a two-week holdback test: randomly assign 15% of new signups to skip the email sequence while keeping everything else identical, then compare 30-day retention between the holdback and recipient groups within the same calendar window. This isolates the sequence's effect from seasonality, channel mix, and concurrent changes in a fraction of the time a full RCT would take, since it only needs to resolve the sign and rough magnitude of the effect, not a precise estimate.
Section 5: Verdict
Investigate further before acting. The 12-point retention gap is large enough to be worth pursuing, but at least three of the six alternative explanations (channel mix shift, concurrent changes, and self-selection via completion) are both plausible and cheap to check. Committing to double a team's headcount on an uncontrolled before/after comparison risks a costly reversal if the holdback test shows a smaller or null effect.
A maioria das decisões baseadas em dados falha não porque os dados estavam errados, mas porque ninguém tentou seriamente refutar a primeira explicação que se encaixava. Um número de retenção sobe após um lançamento, e o lançamento recebe todo o crédito, mesmo que três ou quatro outras coisas tenham mudado na mesma janela. Este prompt foi construído para interromper esse padrão antes que uma equipe comprometa orçamento real com uma história causal não verificada.
Por que isso usa uma estrutura de alternativas forçadas
O prompt não pede simplesmente para “analisar estes dados” — esse enquadramento tende a produzir confirmação de qualquer alegação já presente no prompt. Em vez disso, a seção de Tarefa instrui explicitamente o modelo a gerar explicações alternativas primeiro, antes de ser permitido avaliar a alegação original.
Por que o formato de saída usa uma tabela de evidências distintivas, não uma conclusão
Uma lista de explicações alternativas só é útil se você souber como distingui-las. Por isso a Seção 2 emparelha cada explicação alternativa com uma evidência específica e verificável.
Por que recomenda o teste mais barato, não o mais rigoroso
Um ensaio controlado randomizado completo é o padrão-ouro para alegações causais, mas a maioria das equipes não tem tempo ou disposição para executar um para cada métrica que muda. A seção de Restrições instrui explicitamente o modelo a preferir o diagnóstico mais barato que mude significativamente a confiança.
Como usar bem
Preencha os três campos entre colchetes com o máximo de especificidade possível; entradas vagas produzem explicações alternativas vagas. Este prompt funciona melhor quando você já suspeita que a história causal pode ser conveniente demais. Use-o antes que uma decisão de alocação de recursos seja tomada, não depois.