A scale designed to measure 'social anxiety' shows high test-retest reliability and high internal consistency. However, it correlates just as strongly with depression scales as with other anxiety measures. What validity problem does this indicate?
ALow reliability — the scale is not producing consistent results
BPoor discriminant validity — the scale is not distinguishing social anxiety from related but distinct constructs like depression
CLow predictive validity — the scale cannot forecast anxious behavior in future situations
DLow internal validity — the study lacks a control group
Discriminant validity asks whether a measure fails to correlate with measures of different constructs. If an anxiety scale correlates as highly with depression as with other anxiety measures, it may be measuring general negative affect rather than anxiety specifically — a discriminant validity failure. Options C and D confuse measurement validity with design validity. Option A is wrong because high test-retest reliability and internal consistency were explicitly stated.
Question 2 Multiple Choice
A researcher creates a measurement instrument with perfect test-retest reliability and near-perfect internal consistency. She concludes the instrument must be valid because it is highly reliable. What is wrong with this reasoning?
ANothing — high reliability is both necessary and sufficient to establish validity
BReliability is irrelevant to validity; she should not have measured it at all
CReliability is necessary but not sufficient for validity — a consistent measure can systematically measure the wrong thing
DValidity can only be established through experimental randomization, not by checking reliability
The classic analogy: a scale that consistently reads 10 pounds too heavy is perfectly reliable and perfectly invalid. Consistency (reliability) tells you the measure is precise, but not that it is accurate — not that it actually captures the construct it claims to measure. Validity requires additional evidence: convergent validity (does it correlate with other measures of the same construct?), discriminant validity (does it fail to correlate with different constructs?), predictive validity, and content validity. High reliability is a precondition for validity, not proof of it.
Question 3 True / False
A measurement instrument that is highly reliable — producing consistent results across repeated administrations — is therefore also valid for any research use it is applied to.
TTrue
FFalse
Answer: False
Reliability is necessary but not sufficient for validity. A scale that consistently over- or under-measures, or that measures a related but different construct, will be reliable and invalid. Moreover, validity is not a property of an instrument in isolation — it is a property of its use in a specific context. The same instrument may have strong validity evidence in one population and weak validity evidence in another.
Question 4 True / False
Construct validity, internal validity, and external validity are all distinct concepts: construct validity concerns whether the measurement instrument captures the intended theoretical concept, while internal and external validity concern the design and generalizability of the study.
TTrue
FFalse
Answer: True
These three validity types operate at different levels and are frequently confused. Construct validity is a measurement question: does your instrument measure what you say it measures? Internal validity is a causal inference question: can you attribute the observed relationship to the cause you claim, or is it confounded? External validity is a generalization question: do findings from your sample and context apply more broadly? A study can have high construct validity but low internal validity (valid measures, confounded design), or the reverse.
Question 5 Short Answer
Using a concrete analogy, explain why reliability is said to be 'necessary but not sufficient' for validity.
Think about your answer, then reveal below.
Model answer: A scale that consistently reads 10 pounds too heavy is perfectly reliable — it gives the same result every time — but it is invalid as a measure of true body weight because its readings are systematically wrong. Reliability (consistency) is necessary because a measure that gives random results on repeated testing cannot be measuring anything real. But consistency alone only means the measure is precise; it does not mean the measure is accurate — that it captures the intended construct. Validity requires showing not just consistency, but that the consistent scores actually reflect the theoretical concept of interest.
This distinction between precision and accuracy applies throughout social science measurement. Many well-validated intelligence or personality scales are highly reliable. The ongoing debates are about validity: do they measure what they claim? Reliability settles the precision question; validity requires accumulating multiple forms of evidence — convergent, discriminant, predictive, and content validity — that build a cumulative case.