Questions: Classical Test Theory Foundations

Question 1

According to CTT, if you could administer the same test to the same person an infinite number of times under identical conditions, their average observed score would converge to what?

• A. Their score on the first administration
• B. Their true score
• C. The population mean on that test
• D. Zero, because errors accumulate

Answer: (B)

The true score is formally defined in CTT as the expected value (mean) of observed scores over an infinite number of independent administrations. Because measurement errors are assumed to be random with a mean of zero, they cancel out over many trials, leaving only the true score. This definition makes the true score a theoretical construct — never directly observable.

Question 2

In CTT, measurement error is assumed to be systematic — it consistently pushes a person's observed score in the same direction across repeated testings.

• T. True
• F. False

Answer: False

CTT assumes errors are *random*, not systematic. Each testing occasion produces an independent error value drawn from a distribution with mean zero. Systematic errors (e.g., a test that is always too easy for a particular group) violate CTT assumptions and represent validity problems, not mere unreliability. This distinction between random error (reliability) and systematic error (bias/validity) is fundamental to measurement theory.

Question 3

A test has a reliability coefficient of 0.90. What does this tell you about the relationship between observed scores and true scores on this test?

Model answer: A reliability of 0.90 means that 90% of the variance in observed scores is attributable to true score variance, while 10% is measurement error variance. Equivalently, observed scores on this test are highly consistent — a person who takes it twice under similar conditions will likely get very similar scores. High reliability is necessary (but not sufficient) for a test to be useful.

In CTT, reliability is defined as the ratio of true score variance to observed score variance: r = σ²_T / σ²_X. Since σ²_X = σ²_T + σ²_E, a reliability of 0.90 means error accounts for only 10% of score variance. This is important because a test can only be valid if it is first reliable — you cannot measure what you intend to measure if scores are dominated by random noise.