Questions: Measurement Error and Attenuation of Effects

Question 1

A study finds an observed correlation of r = .20 between anxiety and academic performance. The anxiety scale has reliability .64 and the performance measure has reliability .81. What does the disattenuation formula estimate as the true correlation?

• A. ~.20 — random error does not change the magnitude of observed correlations
• B. ~.28 — the true relationship is larger than the attenuated observed correlation
• C. ~.14 — attenuation inflates observed correlations above the true value
• D. Cannot be determined without knowing the sample size

Answer: (B)

r_true = r_observed / √(r_xx × r_yy) = .20 / √(.64 × .81) = .20 / √(.5184) = .20 / .72 ≈ .28. The true correlation is larger because random error in both measures dilutes the observed relationship. Option A reflects the core misconception — that noise is negligible. Option C has the direction backwards; attenuation shrinks observed correlations below the true value, never inflates them.

Question 2

A researcher runs a study and fails to find a significant effect. She plans to replicate it with three times the sample size. What will this accomplish?

• A. It will recover the true effect by reducing attenuation in the measures
• B. It will increase power to detect the already-attenuated effect, but the attenuated estimate remains unchanged
• C. It will eliminate the random measurement error that caused the null result
• D. It will make the study even more underpowered because larger N reveals more noise

Answer: (B)

Attenuation is a property of the measurement instruments, not the sample. Tripling N increases statistical power — the ability to detect whatever effect exists in the data — but the observed effect is already attenuated by measurement unreliability. The true effect is .40 but the study is estimating .26 (or similar); a larger sample will more precisely estimate .26, but cannot raise it to .40. Only better instruments (higher reliability) can do that.

Question 3

Improving a scale's reliability from α = .64 to α = .81, with all else held equal, would increase the proportion of the true correlation that appears in the observed data.

• T. True
• F. False

Answer: True

The observed correlation equals the true correlation × √(r_xx × r_yy). Raising r_xx from .64 to .81 increases the multiplier from √.64 = .80 to √.81 = .90, recovering more of the true correlation. This is the direct mechanism by which better measurement reduces attenuation.

Question 4

A researcher can fully compensate for low measurement reliability (α = .60) by doubling the sample size, because larger N reduces the impact of random error.

• T. True
• F. False

Answer: False

This is the central misconception about attenuation. Doubling N increases the precision of the attenuated estimate but does not change the ceiling: the observed effect is still shrunk by the factor √(r_xx × r_yy). The only way to recover the true effect size is to improve the reliability of the measures themselves. Adding participants helps you find the attenuated effect more reliably, but you are still measuring a shrunken version of reality.

Question 5

A study finds a null result — no significant relationship between conscientiousness and job performance. How could measurement attenuation explain this, and what is the only way to recover the true effect?

Model answer: Attenuation could explain the null result if the measures had low reliability. Even a substantial true relationship (e.g., r = .40) can appear near zero in observed data if both instruments are noisy — e.g., two measures with reliability .49 would attenuate the observed correlation to only .40 × .49 = .196. The study would be underpowered for that attenuated estimate and could easily miss it entirely. The only way to recover the true effect size is to improve measurement reliability — using validated instruments, increasing the number of items, or improving standardization. No increase in sample size can raise the ceiling set by measurement quality.

This tests whether students understand that null results can be false negatives caused by noisy instruments rather than absent effects. The disattenuation formula makes this precise: it allows researchers to estimate what the true relationship would be with perfect measurement. The practical implication is that measurement quality is a design constraint that cannot be patched with more data.