Questions: Construct Validity and Operationalization of Psychological Constructs

Question 1

A researcher develops an anxiety questionnaire. Participants take it twice, two weeks apart, and get nearly identical scores. The scores also correlate strongly with depression measures and moderately with general neuroticism measures. What validity problem does this reveal?

• A. Poor reliability — the scores should differ more across occasions to be valid
• B. Poor discriminant validity — the measure may be capturing general negative affect rather than anxiety specifically
• C. Poor convergent validity — the measure should correlate with more anxiety-related outcomes
• D. No problem — high reliability and correlations with related constructs are exactly what is needed

Answer: (B)

High reliability is good, but the pattern of correlations suggests a discriminant validity problem. If an 'anxiety' measure correlates just as strongly with depression as with other anxiety measures, it may be measuring something broader — general negative affect or neuroticism — rather than anxiety specifically. Construct validity requires not only convergent evidence (correlating with theoretically related things) but also discriminant evidence (not correlating too strongly with theoretically distinct constructs). The high test-retest reliability actually makes the discriminant problem more concerning, not less.

Question 2

A psychologist argues that their self-report intelligence test is valid because it produces highly consistent scores across administrations (Cronbach's alpha = .95). What is the most important flaw in this reasoning?

• A. Self-report measures cannot be valid for intelligence; only performance tasks qualify
• B. Reliability establishes consistency but not whether the measure actually captures the construct of intelligence
• C. A Cronbach's alpha of .95 is too high and indicates item redundancy
• D. Construct validity requires multiple administrations, not internal consistency

Answer: (B)

Reliability is necessary but not sufficient for construct validity. A perfectly reliable measure can consistently measure the wrong thing — as the Explainer puts it, a ruler reliably measures length but is an invalid measure of intelligence. High Cronbach's alpha tells you items are internally consistent; it says nothing about whether they converge on the right construct. Demonstrating construct validity requires convergent evidence (correlating with other intelligence measures), discriminant evidence (not over-correlating with unrelated constructs), and theoretical grounding for what the items are designed to capture.

Question 3

A measure of 'academic motivation' validated with Western university students can be applied directly to children in different cultural settings without re-establishing validity.

• T. True
• F. False

Answer: False

Construct validity is context-dependent — it must be re-established when a measure is applied to new populations, cultures, or settings. The social meaning of academic performance, the relevance of specific items, and the factor structure of a construct can all differ substantially across cultural contexts. A measure validated in one setting provides no guarantee of validity elsewhere. This is why validation is an ongoing program of research, not a one-time certification, and why cultural sensitivity in measurement is an empirical, not merely ethical, requirement.

Question 4

Using three different methods to operationalize the same construct — behavioral observation, self-report, and physiological measurement — and finding that all three converge on similar conclusions provides stronger construct validity evidence than any single method.

• T. True
• F. False

Answer: True

This is the core logic behind the multitrait-multimethod matrix and the broader principle of convergent validation. When multiple operationalizations using different methods all point in the same direction, it becomes less likely that the results are artifacts of one particular measurement approach. The convergence across methods separates genuine construct effects from method-specific variance, providing stronger evidence that observed effects reflect the underlying construct rather than the peculiarities of any single measurement procedure.

Question 5

Explain why a measure can be perfectly reliable yet have poor construct validity. Give an example.

Model answer: Reliability means a measure consistently produces the same scores; construct validity means it captures the intended theoretical construct. These are independent: a ruler is perfectly reliable (consistent) but invalid as a measure of intelligence. In psychology, a highly internally consistent scale might reliably measure 'agreeableness with strangers' when the researcher intends to measure 'empathy' — consistent, but systematically off-target. The measure would pass reliability checks while having construct-irrelevant variance (capturing something other than empathy) and construct underrepresentation (missing important dimensions of empathy). Reliability rules out random error; construct validity rules out systematic error in the choice of what to measure.

The asymmetric relationship — reliability is necessary but not sufficient for validity — is the key point. Random measurement error undermines both reliability and validity simultaneously, so eliminating it (high reliability) is a prerequisite. But a systematic error, like measuring the wrong construct consistently, is undetectable by reliability analysis alone. This is why construct validation requires additional evidence beyond consistency: convergent correlations, discriminant correlations, theory-driven predictions, and cross-cultural generalization tests.