Questions: Assumption Violations and Statistical Test Robustness

Question 1

A researcher measures anxiety in 50 participants, asks the same 50 participants to complete a stress task, then measures anxiety again. She treats all 100 anxiety scores as 100 independent observations in a t-test. What is the primary statistical problem?

• A. The sample size of 50 is too small to use a t-test
• B. Two measurements from the same person are correlated, so observations are not independent — the effective sample size is much smaller than 100
• C. Anxiety scores are likely non-normal, which invalidates the t-test
• D. She should have used an ANOVA rather than a t-test for this design

Answer: (B)

Independence is violated: two scores from the same person will be correlated (anxious people tend to be anxious on both occasions). When correlated observations are treated as independent, the standard error is underestimated, artificially inflating the t-statistic and the false positive rate. Treating 50 paired observations as 100 independent ones can dramatically inflate Type I error. Non-normality (option C) is generally a lesser concern, especially with larger N. Option D (ANOVA) is not the issue — the independence violation is the same regardless of which test is used.

Question 2

A t-test is described as 'robust to non-normality.' What does this most precisely mean?

• A. The p-value is identical whether or not the normality assumption holds
• B. The test can be applied to any data distribution without any loss of power
• C. The Type I error rate stays close to the nominal alpha level even when normality is violated, especially with larger samples
• D. Non-normality only matters for t-tests when sample sizes are very small

Answer: (C)

Robustness means the test's operating characteristics — primarily its Type I error rate — remain close to their intended values despite assumption violations. It does NOT mean p-values are unchanged (option A) or that there is no cost to power (option B). The Central Limit Theorem explains why robustness to non-normality improves with larger samples: sampling distributions of means become approximately normal regardless of the underlying distribution. Option D overstates the case — robustness applies broadly with moderate to large equal-sized groups, not just very small samples.

Question 3

A statistical test that is robust to an assumption violation produces the same p-value as it would if that assumption were perfectly satisfied.

• T. True
• F. False

Answer: False

Robustness means the Type I error rate (and ideally power) stays close to nominal despite the violation — not that p-values are numerically identical under both conditions. The p-value itself may differ when assumptions are violated; what robustness guarantees is that the false positive rate remains approximately controlled at the alpha level. Confusing 'robust' with 'unaffected' leads to the mistaken belief that robustness makes assumption checking irrelevant.

Question 4

With sufficiently large and roughly equal group sizes, the independent-samples t-test remains approximately valid even when the normality assumption is violated.

• T. True
• F. False

Answer: True

The Central Limit Theorem ensures that sampling distributions of means approach normality as sample size increases, regardless of the shape of the underlying distribution. For the t-test, this means the test statistic follows approximately the expected distribution even with non-normal data, provided group sizes are reasonably large (often cited as n ≥ 30 per group as a rough heuristic) and roughly equal. This is why normality violations are generally more forgiving than independence violations — the latter have no such CLT rescue.

Question 5

Why is violating the independence assumption generally considered more serious than violating the normality assumption for parametric tests like the t-test?

Model answer: Independence violations inflate the false positive rate in ways the Central Limit Theorem cannot fix, because clustered observations carry less information than truly independent ones — the effective sample size may be far smaller than the nominal N.

When observations are correlated (e.g., multiple responses from the same person, students within classrooms), the standard error is computed as if observations were independent, which underestimates true sampling variability. The resulting t-statistic is inflated, and the p-value is too small. This can make truly null results appear significant at rates far exceeding the nominal alpha. Normality violations, by contrast, are largely rescued by the Central Limit Theorem as N grows — the sampling distribution of means normalizes regardless of the population shape. There is no analogous rescue for non-independence; the structural problem remains no matter how large N becomes.