What intervention changes?
Define X precisely.
A vague exposure creates a vague causal question.
Side 37
Causal
Inference
A study of the missing comparison. Causal inference asks what would have happened to the same unit under a different intervention, then builds research designs that approximate that unobservable counterfactual.
intervention→counterfactual→identification→estimate→sensitivity
06design families
05bias threats
05diagnostic questions
37Side
Causality compares potential outcomes.
For one unit, we can observe the outcome under treatment or control, never both at the same time.
What response matters?
Define Y and timing.
Outcomes must be measured after the intervention on a meaningful horizon.
What would have happened otherwise?
The missing potential outcome.
The fundamental causal problem is that this comparison is unobservable for the same unit.
What design recovers the comparison?
Randomization or assumption?
Identification connects observed data to the causal estimand.
How large is the effect?
Average, local, heterogeneous?
Causal effects can vary across units and contexts.
DAGs turn causal assumptions into visible structure.
Directed acyclic graphs help distinguish confounders, mediators and colliders before statistical adjustment begins.
Common cause of treatment and outcome.
Failing to control a true confounder can create biased treatment comparisons.
Lies on the causal pathway.
Controlling a mediator changes the causal question from total effect to something narrower.
Common effect of two variables.
Conditioning on a collider can create spurious association.
Noncausal route between treatment and outcome.
Adjustment aims to block appropriate backdoor paths without opening new ones.
Moves treatment through a restricted path.
Instrumental variables require strong assumptions about relevance and exclusion.
Who enters the sample?
Conditioning on selection processes can distort causal relationships.
Randomization manufactures comparability in expectation.
By making treatment assignment independent of baseline characteristics, randomized experiments simplify causal identification.
Randomize
Break systematic treatment selection.
Large randomized groups should be comparable on both observed and unobserved pre-treatment factors on average.
Control
Define the counterfactual condition.
The control condition determines what causal contrast the experiment actually estimates.
Compliance
Assignment may differ from treatment received.
Intention-to-treat and treatment-on-the-treated answer different questions.
Interference
One unit may affect another.
Spillovers violate simple assumptions that each unit’s outcome depends only on its own assignment.
Generalize
Internal validity is not external validity.
A clean experiment can still fail to travel to another population or setting.
Observational designs seek quasi-experimental contrasts.
When randomization is unavailable, credibility comes from design logic plus assumptions that make comparison groups plausibly informative.
| Design | Identification idea | Key assumption | Typical use |
|---|---|---|---|
| Matching / weighting | Balance observed covariates | No important unmeasured confounding | Comparable treated and untreated groups |
| Difference-in-differences | Compare changes over time | Parallel trends absent treatment | Policy or shock affecting one group |
| Regression discontinuity | Compare units near a threshold | No precise manipulation at cutoff | Eligibility rules |
| Instrumental variables | Use exogenous treatment shifter | Exclusion + relevance + monotonicity variants | Endogenous treatment choice |
| Synthetic control | Construct weighted comparison unit | Pre-treatment fit captures relevant trajectory | Single treated region or organization |
Bias enters when the comparison stops representing the counterfactual.
Most causal arguments fail through design assumptions, not through arithmetic.
Treatment groups differ before treatment.
Shared causes can mimic treatment effects.
Outcome influences exposure.
Temporal ordering can invert the apparent causal direction.
Treatment or outcome is mismeasured.
Error can attenuate, distort or create associations depending on structure.
Follow-up differs by group.
Selective dropout can destroy original comparability.
Adjusting for a consequence changes the estimand.
Conditioning on mediators or colliders can bias estimates.
Treatment crosses group boundaries.
Interference can contaminate controls and alter interpretation.
Causal clinic.
Ask what comparison would make the causal sentence credible.
“Training increased sales.”
Compare trained and untrained salespeople after accounting for why some received training. Better: randomize access or exploit a credible rollout design. Avoid comparing volunteers with non-volunteers as though selection were irrelevant.
“The new policy reduced accidents.”
Look at pre-policy trends, simultaneous changes, comparable untreated units and whether reporting practices changed at the same time.
“Customers who use feature X retain longer.”
Usage may be a marker of already-engaged customers. Estimate what happens when comparable customers are induced or randomized to use the feature.
“Higher hospital volume causes better outcomes.”
Volume may improve expertise, but better hospitals may also attract more patients. Directionality and referral selection must be separated.
Causal Inference: The MixtapeScott Cunningham · applied designs
The Book of WhyPearl & Mackenzie · causal graphs
Causal Inference: What IfHernán & Robins · potential outcomes
Mostly Harmless EconometricsAngrist & Pischke · quasi-experimental methods