A hospital sets an extremely strict diagnostic threshold for a rare disease (very low α, so only the most extreme test results trigger a positive diagnosis). What is the most likely consequence?
AFewer false positives AND fewer false negatives, since the strict threshold makes the test more accurate overall
BMore false positives, because the strict threshold makes the test oversensitive
CMore false negatives (missed cases), because the smaller rejection region is harder to enter even when the disease is present
DNo effect on false negatives — α only controls false positives
Lowering α shrinks the rejection region (moves the critical value further into the tail). This reduces false positives (good) but simultaneously pushes more of the alternative distribution H₁ into the non-rejection region — increasing β, the false negative rate. In the medical context: a test that almost never calls a healthy person sick (low α) will frequently miss patients who are genuinely sick (high β). The tradeoff is unavoidable for a fixed sample size.
Question 2 Multiple Choice
A research team wants to simultaneously achieve α = 0.01 (very strict significance) and 0.95 power (very high sensitivity) without collecting additional data. Is this feasible?
AYes — choosing the optimal test statistic can eliminate the tradeoff between α and power
BYes — switching from a two-tailed to a one-tailed test automatically achieves both goals
CNo — for a fixed sample size and effect size, reducing α necessarily increases β and reduces power
DNo — once α is chosen, power is fixed regardless of sample size or effect size
For a fixed sample size and effect size, α and β are in direct tension: shrinking the rejection region to achieve α = 0.01 pushes more of the H₁ distribution into the non-rejection region, lowering power. The only way to achieve both strict α and high power is to collect more data (larger n), which narrows the sampling distributions under both H₀ and H₁, making them easier to separate.
Question 3 True / False
Increasing sample size is the only design lever that can simultaneously reduce the Type I error rate and increase statistical power.
TTrue
FFalse
Answer: True
For a fixed effect size and test design, α and β are linked: reducing one increases the other. Increasing sample size narrows the sampling distributions under both H₀ and H₁, allowing a stricter critical value (lower α) while keeping most of the H₁ distribution in the rejection region (lower β, higher power). Effect size is a property of reality, not a design choice; sample size is the primary lever the experimenter controls for reducing both error rates simultaneously.
Question 4 True / False
A test with significance level α = 0.01 is more statistically powerful than a test with α = 0.05, most else being equal.
TTrue
FFalse
Answer: False
This is exactly backwards. A stricter significance level (α = 0.01) means a smaller rejection region — the critical value is pushed further into the tail. For the same sample size and effect size, more of the H₁ distribution falls in the non-rejection region, so β increases and power (= 1 − β) *decreases*. The test with α = 0.05 has more power. Stricter α makes it harder to reject H₀ — which hurts sensitivity, not helps it.
Question 5 Short Answer
Explain why reducing the significance level α necessarily increases the Type II error rate β, for a fixed sample size and effect size.
Think about your answer, then reveal below.
Model answer: Reducing α means making the rejection region smaller by moving the critical value further into the tail of the H₀ distribution. But the H₁ distribution overlaps with the H₀ distribution — the amount of overlap is fixed by sample size and effect size. A smaller rejection region means less of the H₁ distribution falls inside it, so the probability of correctly detecting a true effect (power) decreases and β = 1 − power increases. There is no free adjustment that shrinks both error rates without narrowing the distributions — which only larger n achieves.
Geometrically: imagine two overlapping bell curves, one centered at the null value and one at the true parameter value. The rejection region is the right tail past the critical value. Moving the boundary rightward (lower α) cuts off part of the alternative distribution that previously fell in the rejection region — that cut-off portion becomes additional β. The only fix is to make the curves narrower (larger n) or more separated (larger effect size, which isn't a design choice).