Questions: Content Validity and Domain Representation

Question 1

A nursing licensure exam is found to have high internal consistency reliability (α = 0.92) but covers only factual recall of pharmacology, omitting clinical reasoning, patient communication, and emergency procedures. What is the primary validity problem?

• A. The exam lacks construct validity because its factor structure is poorly defined
• B. The exam lacks content validity because it fails to adequately sample the full domain of nursing competence
• C. The exam lacks criterion validity because it is not correlated with actual nursing performance
• D. The exam lacks reliability because different items measure different things

Answer: (B)

Content validity is about whether the test items adequately sample and proportionally represent the full domain. Nursing competence includes pharmacology, but also clinical reasoning, patient assessment, communication, and emergency response. A test that covers only one narrow subdomain has poor content validity regardless of how internally consistent those items are. High reliability tells you the test is measuring something consistently — it doesn't tell you whether what it's measuring covers the right ground. This is the domain sampling problem.

Question 2

A test developer wants to establish content validity for a new certification exam. What is the primary tool for doing so?

• A. Computing the correlation between exam scores and a gold-standard external criterion
• B. Running a factor analysis to confirm the test measures a single underlying construct
• C. Convening subject matter experts to evaluate whether items adequately represent the domain
• D. Administering the test to a large sample and checking for floor and ceiling effects

Answer: (C)

Content validity rests on expert judgment and logical analysis, not statistical indices. The standard process involves defining a content blueprint (table of specifications) and then having subject matter experts evaluate each item for relevance and representativeness. Tools like the content validity ratio (CVR) quantify the degree of expert agreement, but they organize expert judgment rather than replace it. The other options describe criterion validity (correlation with external criterion), construct validity (factor analysis), and item quality checks — not content validity.

Question 3

Content validity can be fully established before data are collected, because it is a logical and judgmental question rather than a statistical one.

• T. True
• F. False

Answer: True

This is one of the key features that distinguishes content validity from other forms of validity evidence. Criterion validity and construct validity require data — correlations with outcomes, factor analyses, convergent/discriminant patterns. Content validity, by contrast, is evaluated by examining the logical relationship between test items and the domain being measured, through structured expert judgment. The content blueprint and expert review process can happen entirely during test development, before a single examinee is tested. This makes content validity the most foundational validity consideration in test design.

Question 4

A test that demonstrates excellent content validity — items that proportionally cover the full domain — is very likely to be a valid measure of the underlying construct.

• T. True
• F. False

Answer: False

Content validity is necessary but not sufficient for overall validity. Even if every item covers the right content area, poorly written items might measure something other than substantive knowledge — for example, reading comprehension rather than the target domain. An item that covers the correct clinical nursing content but is written in needlessly complex language may be measuring verbal ability as much as nursing knowledge. Content validity addresses the sampling question (is the right content covered?), not the measurement question (do the items actually elicit the target knowledge?). Full validity evidence requires multiple sources.

Question 5

What is the domain sampling logic of content validity, and why does it mean content validity is a judgment about the test as a whole rather than about individual items?

Model answer: Content validity treats the test as a sample drawn from a larger universe of possible questions about the construct. A test has good content validity if that sample is representative — if the items cover the major areas of the domain in proportions that reflect their importance, rather than concentrating on easy-to-measure or frequently-tested subsets. This makes it a judgment about the collection of items as a whole: even if every individual item is excellent, if the collection fails to cover key areas of the domain, the test has poor content validity. A nursing exam that omits clinical reasoning is deficient as a sample even if its pharmacology items are flawless.

The sampling metaphor is useful because it clarifies what content validity is and is not. It is not about whether individual items are well-written (item quality) or statistically well-behaved (psychometric properties). It is about coverage — whether the test systematically samples the domain in a way that reflects its actual structure and priorities. This is why the content blueprint or table of specifications is central to content validity: it defines what the domain looks like so you can evaluate whether the sample is representative of it.