Classical test theory posits that an observed score comprises a true score plus random error. This framework provides methods to estimate reliability and understand score variance distributions. CTT focuses on total test score analysis and is foundational for understanding measurement precision and error.
Every time a person takes a test, their score is influenced by more than just what they actually know or how able they actually are. They may be tired, distracted by a question's wording, lucky on a guess, or unlucky on a difficult problem that they would usually solve. Classical test theory (CTT) is a mathematical framework that formally acknowledges this and gives us tools to reason about measurement precision despite it.
The central model is elegantly simple: X = T + E, where X is the observed score, T is the true score, and E is the random error. The true score is a theoretical construct — the score a person would get if measurement error averaged out completely, which you can think of as the long-run average over infinitely many independent testings. The error term E captures all the random, unsystematic factors that make any single administration noisy. Crucially, CTT assumes E has a mean of zero across repeated measurements: positive and negative errors cancel. This means your observed score on any given day is an unbiased estimate of your true score, just with noise added.
The practical payoff of this framework is the concept of reliability: the proportion of observed score variance that reflects true score variance rather than error. Formally, reliability ρ = σ²_T / σ²_X. A reliability of 1.0 would mean every observed score perfectly reflects the true score (no error at all). A reliability of 0 would mean observed scores are pure noise. In practice, well-constructed psychological tests aim for reliabilities of 0.80 or higher. Your statistics background is directly relevant here: variance decomposition (σ²_X = σ²_T + σ²_E) is the mathematical engine underlying CTT.
A critical distinction — one that CTT specifically cannot handle well — is between *random* error (which CTT models) and *systematic* error (which it does not). If a test is consistently harder for one demographic group due to biased item content, CTT's reliability coefficient will not detect this; the test may appear reliable while being systematically unfair. This limitation motivates more modern approaches like item response theory (IRT), but CTT remains the essential starting point for understanding what measurement precision means and how to quantify it.
The concepts you will encounter next — reliability-validity relationships and item difficulty/discrimination — extend directly from this foundation. Reliability is a necessary condition for validity (a noisy test cannot measure anything well), but not sufficient (a consistent test could still measure the wrong thing). Item-level analysis then lets you diagnose *which* specific questions contribute to error and which ones are doing the most work in distinguishing true score differences among test-takers.