Questions: Hypothesis Testing in Regression

Question 1

A regression on n = 1,000,000 observations finds β̂₁ = 0.0004 with a p-value of 0.001. Which conclusion is most defensible?

• A. The result is both statistically and economically significant
• B. The result is statistically significant but the effect size is almost certainly economically trivial
• C. The result is not significant because the coefficient is very small
• D. The p-value is meaningless with such a large sample

Answer: (B)

With a million observations, standard errors are tiny — even minuscule effects produce significant p-values. A coefficient of 0.0004 means a one-unit change in x predicts a 0.04% change in y. Whether that matters economically depends on the context, but it is almost never substantively important. Statistical significance tells you the effect is probably real; economic significance tells you whether it matters.

Question 2

A p-value of 0.04 means there is a 4% chance that the null hypothesis is true.

• T. True
• F. False

Answer: False

The p-value is the probability of observing a test statistic as extreme as the one computed, *assuming the null hypothesis is true*. It is not the probability that H₀ is true. A p-value of 0.04 means: if H₀ were true, you would see a result this extreme 4% of the time. Inverting this — saying H₀ has a 4% chance of being true — is a common and serious misinterpretation.

Question 3

A researcher tests H₀: β₁ = 0 and gets a p-value of 0.42. Does this prove the true coefficient is zero? Explain.

Model answer: No. Failing to reject H₀ does not prove it is true. The result may simply reflect low statistical power — a sample too small or variance too high to detect a real but modest effect.

Hypothesis testing can only reject or fail to reject the null — it cannot confirm it. A p-value of 0.42 means the data are consistent with β₁ = 0, but they are also consistent with many nonzero values. This is especially true when sample size is small, since confidence intervals will be wide and include zero even for economically meaningful effect sizes.