Questions: Internal Validity and Threats to Causal Inference

Question 1

A researcher tests whether a mindfulness program reduces student anxiety. All participants complete the 8-week program during finals season. Anxiety is measured before (high-stress exam period) and after (summer break). Students show significantly lower post-program anxiety. What is the main threat to internal validity?

• A. Regression to the mean — anxious students naturally return to baseline
• B. Selection bias — the students who enrolled chose to be there
• C. History or maturation — the exam season ending could explain the anxiety decrease, not the program
• D. Instrumentation — the anxiety measure may not be valid

Answer: (C)

This is a textbook history/maturation confound. The pre-post change coincides with moving from finals week (high-stress) to summer (low-stress). Any improvement in anxiety could be caused by this external event (history) or natural recovery once the stressor passes (maturation), not the mindfulness program. There is no control group experiencing the same time period without the treatment, so we cannot distinguish the treatment effect from the passage of time.

Question 2

A study targets struggling readers (bottom 10% on a pretest), implements a reading intervention, and finds their posttest scores improved significantly. A skeptic says this may not reflect a real treatment effect. Why?

• A. Struggling readers cannot improve regardless of intervention, making the gains suspicious
• B. Regression to the mean: extreme low scorers tend to score closer to the population average on a second test regardless of any treatment, because extreme scores partly reflect measurement error
• C. Selection bias: the students chose to participate, making them unrepresentative
• D. The study lacks random assignment, so no conclusions are possible

Answer: (B)

Regression to the mean is the key threat here. When participants are selected because they scored extremely low on a pretest, their true scores are near the floor, but their measured scores include downward measurement error. On a second measurement, that error is less likely to be extreme and downward, so scores tend to drift upward toward the group mean — regardless of any intervention. Studies targeting 'at-risk' populations without a parallel control group systematically overestimate treatment effects for this reason.

Question 3

Adding a no-treatment control group to a study helps rule out history and maturation as alternative explanations for observed changes in the treatment group.

• T. True
• F. False

Answer: True

A control group experiencing the same time period, environment, and measurement procedures as the treatment group but not receiving the treatment controls for history (both groups face the same external events) and maturation (both groups age and develop at the same rate). If the treatment group improves more than the control group, the difference is less likely to be explained by history or maturation alone. This is why the pre-post-with-control design is fundamentally stronger than a simple pre-post design.

Question 4

Random assignment to conditions eliminates most major threats to internal validity, making additional experimental controls unnecessary.

• T. True
• F. False

Answer: False

Random assignment is powerful but limited: it controls selection bias by distributing known and unknown confounds equally across conditions at the start. It does not eliminate history (external events can affect both groups differentially if they are tested at different times), maturation (both groups mature, but differential maturation remains if groups experience the study differently), or testing effects (all participants who take a pretest are affected by it). Random assignment is necessary but not sufficient for ruling out all threats to internal validity.

Question 5

What is the key function of a control group in an experiment, framed in terms of threats to internal validity rather than statistical power?

Model answer: The control group serves as a comparison condition that experiences all the same potential confounds as the treatment group — the same passage of time (history and maturation), the same measurement procedures (testing and instrumentation effects) — but without the treatment. Any change observed in both groups can be attributed to these shared alternative explanations. Only the difference between groups can plausibly be attributed to the treatment. Without a control group, you cannot separate the treatment effect from the confounds; the control group is the systematic elimination of rival hypotheses.

Framing the control group in terms of threats reveals its deeper purpose: it is not just a statistical baseline but a structural defense against alternative explanations. Every major threat to internal validity is a rival hypothesis — the control group is what allows you to rule out those rivals. This is why quasi-experimental designs that lack equivalent control groups require much more careful threat-by-threat analysis to support causal conclusions.