Questions: Hypothesis Testing Fundamentals

Question 1

A researcher gets p = 0.03 with α = 0.05. What is the correct interpretation?

• A. There is a 3% probability that H₀ is true
• B. There is a 3% probability that the result occurred by chance if H₀ is true
• C. The alternative hypothesis is 97% likely to be correct
• D. H₀ has been proven false

Answer: (B)

The p-value is P(data this extreme or more | H₀ true). It says nothing about the probability that H₀ is true or false — that would require a Bayesian framework. Options A and C confuse the conditional direction; option D overreaches because statistical tests never prove hypotheses.

Question 2

Failing to reject the null hypothesis (p > α) is equivalent to accepting the null hypothesis as true.

• T. True
• F. False

Answer: False

Failing to reject H₀ means only that we did not find sufficient evidence against it — not that H₀ is true. The test may have low power (small sample, small effect size), meaning a real effect existed but the test could not detect it. 'Absence of evidence is not evidence of absence.'

Question 3

Why do statisticians choose a significance level α before collecting data, rather than after seeing the p-value?

Model answer: Setting α in advance prevents p-hacking — the practice of adjusting the threshold after seeing the data to make a result appear significant. Pre-registration keeps the decision rule honest and ensures the false positive rate (Type I error) is controlled at the intended level.

If α were chosen after observing p, a researcher could always pick α just above the observed p-value to claim significance. Prespecifying α is a form of scientific discipline that separates confirmatory testing (fixed rules) from exploratory analysis.