Questions: True Experimental Design and Randomization

Question 1

A researcher randomly assigns 80 participants to a meditation condition or a control condition to test whether meditation reduces anxiety. A critic notes she never measured participants' prior meditation experience. Is this a valid concern?

• A. Yes — without measuring prior experience, it could confound the results and invalidate the study
• B. No — random assignment distributes prior experience approximately equally across conditions, so it is controlled for even without being measured
• C. Yes — any unmeasured variable is a fatal flaw in a true experiment
• D. No — prior experience is irrelevant because both groups still received the same instructions

Answer: (B)

This is the key insight of randomization. Because participants were randomly assigned, prior meditation experience — along with every other variable, measured or not — is distributed approximately equally across conditions by chance. The critic's concern would be valid in a non-randomized study, but random assignment controls for all confounds simultaneously, including variables the researcher never thought to measure.

Question 2

What is the critical distinction between 'random sampling' and 'random assignment'?

• A. Random sampling determines which condition participants enter; random assignment determines who is recruited for the study
• B. Random sampling selects who participates in the study from a population; random assignment determines which condition participants are placed into
• C. They are equivalent procedures applied at different stages of analysis
• D. Random assignment is used in observational studies; random sampling is used in true experiments

Answer: (B)

Random sampling (drawing a representative sample from a population) is about external validity — generalizing results to the broader population. Random assignment (allocating participants to conditions by chance) is about internal validity — ensuring groups are equivalent before manipulation so that differences in outcomes can be attributed to the treatment. A study can have one without the other.

Question 3

Randomization in a true experiment guarantees that the experimental and control groups are identical on most variables before the treatment begins.

• T. True
• F. False

Answer: False

Randomization ensures group equivalence in expectation — on average, across many replications. With any finite sample, chance variation will leave some imbalance between groups. This is especially true with small samples, which is why statistical testing is still necessary even in randomized experiments: the test checks whether the observed difference between conditions exceeds what random variation alone would produce.

Question 4

Only true experiments with random assignment can straightforwardly support causal inference, because only randomization ensures that groups were equivalent before the treatment was administered.

• T. True
• F. False

Answer: True

In a randomized experiment, the groups are equivalent in expectation across all variables — including unmeasured ones — before manipulation begins. Any subsequent difference in outcomes can therefore be attributed to the treatment. Non-randomized designs (quasi-experiments, observational studies) must use matching, regression adjustment, or other techniques to approximate this equivalence, but they can never be certain they've controlled for all confounds.

Question 5

Why does random assignment allow causal inference in ways that non-randomized designs cannot, even when those designs carefully measure and statistically control for many known confounding variables?

Model answer: Random assignment controls for all confounders simultaneously — including unmeasured and unknown ones — by distributing them equally across conditions by chance. Non-randomized designs can only control for confounders the researcher identifies and measures. There may always be additional unmeasured variables driving the observed association, so non-randomized designs cannot rule out alternative explanations in the way a randomized experiment can.

This is the gold-standard logic of the true experiment. No matter how carefully a non-randomized study measures and adjusts for confounds, a critic can always point to an unmeasured variable that might explain the result. Random assignment eliminates this objection: before treatment, the groups were equivalent by design — not by assumption or measurement.