A researcher uses DiD to evaluate a new public health program introduced in California in 2015 but not in Nevada. Pre-intervention, California's health outcome was improving faster than Nevada's. After applying DiD, the estimate is positive. What is the most serious problem with this analysis?
AThe researcher should have used a randomized controlled trial instead
BThe parallel trends assumption is violated — pre-period trends already diverged, so DiD cannot isolate the program's effect
CDiD requires that both states have the same baseline outcome level before the intervention
DThe estimate is biased because California's larger population makes the comparison unfair
DiD's validity rests entirely on parallel trends: in the absence of the intervention, both groups would have changed at the same rate. If pre-period trends already diverged (California improving faster), then even without the program, California's outcome would have continued improving relative to Nevada. The DiD estimate would conflate this pre-existing trend difference with the program's effect. Baseline levels do not need to be equal — only the *rates of change* must match. Option A confuses the method's purpose (DiD is specifically designed for settings where RCTs are infeasible).
Question 2 Multiple Choice
Using DiD, a researcher finds that a smoking ban in treatment cities reduced hospital admissions by 12 per 10,000 while control cities showed no change. A colleague argues the estimate is confounded because treatment cities have older populations. Why does DiD address this concern — and when does it not?
ADiD removes all confounding; age differences between cities are fully controlled
BDiD removes time-invariant confounders like population age structure, but would be biased if age distributions changed differentially after the ban
CDiD only removes confounders that affect both cities equally, so age differences are not controlled
DDiD requires propensity score matching to control for demographic differences before the estimate is valid
The first difference (post minus pre within each city) removes anything that is constant over time within that city — including stable demographic characteristics like age structure. This is DiD's key advantage over a simple post-intervention comparison. However, if the treatment and control cities' age distributions *diverged* over the study period (perhaps because older residents moved away from treatment cities after the ban), that time-varying confounder is not removed by DiD. The estimator handles time-invariant differences, not time-varying ones.
Question 3 True / False
The parallel trends assumption in DiD can be directly tested by examining pre-intervention data from both groups.
TTrue
FFalse
Answer: True
True — the parallel trends assumption can and should be assessed using pre-intervention data. By examining whether the treatment and control groups showed similar trends *before* the intervention, researchers gain evidence for or against the plausibility of the assumption. Visual plots of pre-period outcomes are the standard diagnostic. Note, however, that this only tests whether the assumption held in the pre-period; it cannot prove the assumption holds in the post-period (that is untestable, because the counterfactual outcome for the treated group is never observed). Pre-trend parallel behavior is necessary but not sufficient evidence.
Question 4 True / False
Difference-in-differences removes both pre-existing level differences and secular trends by subtracting the control group's change from the treated group's change — so a valid DiD estimate requires that the two groups had similar outcome levels at baseline.
TTrue
FFalse
Answer: False
False. DiD does not require baseline level equivalence — it requires parallel *trends*. The first difference (post minus pre within each group) removes whatever is fixed within each group, including their different starting levels. What matters is that both groups would have changed at the same rate absent the intervention. A city with much higher baseline disease rates than the control can still yield a valid DiD estimate if both cities were trending at the same rate before the intervention. Confusing level equivalence with trend parallelism is one of the most common misunderstandings of the method.
Question 5 Short Answer
Why does DiD use two differences rather than one, and what type of confounding does each difference address?
Think about your answer, then reveal below.
Model answer: The first difference — comparing post- to pre-intervention outcomes within each group — removes time-invariant confounders: anything that is constant across time within a group (stable demographics, geography, baseline health culture). The second difference — subtracting the control group's change from the treated group's change — removes common temporal trends: secular changes affecting both groups equally (national health improvements, economic cycles, seasonal patterns). Together, the two differences isolate variation attributable to the intervention itself.
Without the first difference, you compare across groups that may differ in fundamental ways. Without the second difference, you cannot distinguish the program's effect from changes that would have happened anyway. Neither difference alone is sufficient: the first still confounds trends, the second still confounds baseline differences. DiD's power is precisely that the combination eliminates both — provided the parallel trends assumption holds.