Doubly-Definitive Test
A 'doubly-definitive test' provides strong evidence that both confirms a particular causal explanation and rules out other explanations. This type of evidence is rare in evaluations.
Use this test when you need to make a definitive statement about the causal relationship between an intervention and an outcome. It provides the strongest possible support for a causal explanation.
Solves: Need for absolute certainty about the causal relationship, desire to eliminate all alternative explanations, high stakes decision-making.
- 1
Step 1: Identify potential causal factors that could have contributed to the outcome. (10 minutes)
- 2
Step 2: Gather evidence that would both confirm a particular causal explanation and rule out other explanations. (15 minutes)
- 3
Step 3: Assess whether the evidence meets the criteria for a 'doubly-definitive' test. (10 minutes)
- 4
Step 4: Document the rationale for classifying each factor as having or not having 'doubly-definitive' evidence. (5 minutes)
- Recognize that 'doubly-definitive' evidence is rare and may not be available.
- Be extremely rigorous in assessing the evidence and avoid overinterpreting weak evidence.
- Be prepared to acknowledge the limitations of the evidence, even if it is strong.
- Focus on gathering evidence that strongly supports the causal explanation and weakens alternative explanations, even if it doesn't completely rule them out.
- Use a Bayesian approach to quantify the strength of the evidence for and against different causal explanations.