Questions: Multitrait-Multimethod Matrices for Construct Validation

Question 1

An MTMM matrix measures anxiety and depression using self-report and structured interview. The self-report anxiety × self-report depression correlation is r = .72, but the self-report anxiety × interview anxiety correlation (the validity diagonal) is only r = .53. What does this pattern most clearly indicate?

• A. Excellent convergent validity — anxiety and depression are closely related constructs
• B. A method effect — shared measurement variance inflates same-method correlations above cross-method correlations for the same trait
• C. The structured interview has poor reliability and should be replaced
• D. Anxiety and depression are the same construct and should be collapsed into one measure

Answer: (B)

When same-method correlations between different traits (heterotrait-monomethod) exceed cross-method correlations for the same trait (validity diagonal), a method effect is present. Anxiety and depression correlate more highly when both are self-reports than anxiety correlates with itself across methods — meaning the measurement method contributes variance above and beyond the underlying constructs. This is exactly the threat the MTMM design was built to detect: you cannot tell whether correlations reflect shared construct meaning or shared measurement artifact.

Question 2

Which pattern in an MTMM matrix provides the clearest evidence of discriminant validity?

• A. High values in the reliability diagonal, showing each measure is internally consistent
• B. Validity diagonal entries (same trait, different method) are higher than both heterotrait-monomethod and heterotrait-heteromethod correlations
• C. Heterotrait-monomethod correlations being higher than heterotrait-heteromethod correlations
• D. Low correlations throughout the entire matrix, showing the traits are independent

Answer: (B)

Discriminant validity requires that the same trait measured differently correlates more highly with itself than with different traits measured by the same or different methods. The validity diagonal being higher than both types of heterotrait correlations is the critical pattern: the construct is distinguishable from others. If validity diagonal values fall below heterotrait-monomethod values, the measurement method contributes more to observed correlations than the constructs do — a failure of discriminant validity.

Question 3

In a well-designed MTMM matrix, the validity diagonal entries (same trait, different method) should be the highest values in the entire matrix.

• T. True
• F. False

Answer: False

The reliability diagonal — same trait, same method — should contain the highest values, since reliability is the necessary ceiling for validity. Validity diagonal entries should be high, but lower than the reliability estimates. If a validity coefficient exceeded the reliability of the measures involved, that would be mathematically impossible and would signal a data problem. The expected ordering is: reliability > validity diagonal > heterotrait-monomethod > heterotrait-heteromethod.

Question 4

A construct can demonstrate high convergent validity while still failing discriminant validity.

• T. True
• F. False

Answer: True

Convergent validity only shows that a measure correlates with other measures of the same trait. Discriminant validity requires that it does not correlate too highly with measures of different traits. A measure of 'anxiety' might show high cross-method correlations with other anxiety measures (convergent) while also correlating just as highly with depression measures (failing discriminant). This is precisely the situation the MTMM framework was designed to reveal — unidimensional validity evidence systematically overstates construct distinctiveness.

Question 5

Explain why measuring a construct with only one method — even with excellent internal consistency — is insufficient evidence for construct validity in the MTMM framework.

Model answer: Single-method measurement cannot separate trait variance from method variance. A highly internally consistent self-report scale might correlate strongly with other self-report measures not because it measures the intended construct but because all self-reports share common method variance (e.g., social desirability, response style). Without a second method, you cannot distinguish 'this correlates because it measures a related construct' from 'this correlates because we measured it the same way.' MTMM requires cross-method convergence to demonstrate that what is being measured is the construct, not the measurement artifact.

Internal consistency reliability tells you that items cohere within a method — not that the method is measuring the right thing. The MTMM insight is that construct validity requires triangulation across methods: if a construct is real, it should be detectable regardless of how it is measured. Single-method measurement also prevents detecting method effects, which can inflate the apparent validity of measures and lead to overconfident conclusions about what constructs are actually being assessed.