Questions: Item and Test Information Functions and Measurement Precision

Question 1

A test developer wants maximum measurement precision near a pass/fail cut score at θ = 0.5. They select 10 items with high difficulty (b = 2.0), reasoning that harder items are more informative. What does item information theory predict about this choice?

• A. The difficult items provide maximum information at θ = 2.0, far from the cut score — they are poorly targeted and will not improve precision where it matters
• B. Harder items always provide more information because examinees must engage more deeply with them
• C. Item difficulty does not affect where information peaks — only discrimination determines that
• D. The items will provide uniform information across all ability levels, so the choice is acceptable

Answer: (A)

Item information peaks exactly at the item's difficulty parameter b. An item with b = 2.0 is most informative for examinees near θ = 2.0 — well above the cut score of 0.5. For examinees near the cut, this item is easy enough that most of them answer correctly regardless of small differences in ability, contributing little to distinguishing between them. To maximize precision at θ = 0.5, the developer should select items with b values close to 0.5. This is the core principle behind targeted test design and the foundation of computerized adaptive testing.

Question 2

Two items share the same difficulty (b = 0), but item A has discrimination a = 2.0 and item B has a = 0.5. At θ = 0, which statement correctly characterizes their information functions?

• A. Both provide the same information at θ = 0 since they have identical difficulty
• B. Item B provides more information because its gentler slope measures a broader range of ability
• C. Item A provides more information at θ = 0, with a taller and sharper information peak; item B provides less information but spread over a wider range
• D. Discrimination only affects information at extreme theta values, not at the difficulty location

Answer: (C)

The item information function for a 2PL item is proportional to a² · P(θ) · (1 − P(θ)), where P is the probability of a correct response. At θ = b, P = 0.5 and the product P(1−P) is maximized. The a² factor means a high-discrimination item (a = 2.0) provides 16 times the information of a low-discrimination item (a = 0.5) at the same difficulty location. The trade-off is breadth: the high-discrimination item's information curve is narrow and tall, while the low-discrimination item's is wide and flat. For precision at a specific ability level, high discrimination is always preferable.

Question 3

A test designed to make accurate pass/fail decisions near a specific cut score should concentrate items with difficulty values close to that cut score, because item information peaks at the item's difficulty parameter.

• T. True
• F. False

Answer: True

This is the direct design implication of the item information function. Since an item with difficulty b provides maximum information — minimum standard error — for examinees near θ = b, clustering item difficulties around the cut score concentrates measurement precision exactly where the high-stakes decision is made. A licensure exam's main job is to reliably classify examinees as above or below the cut; items targeting other ability levels contribute little to this goal. This principle also explains why computerized adaptive testing is so efficient: it always selects the item maximally informative for the current theta estimate.

Question 4

Classical test theory's single reliability coefficient provides equivalent information about measurement precision as IRT's conditional standard error of measurement — they just express the same thing in different scales.

• T. True
• F. False

Answer: False

The two frameworks differ fundamentally in what they reveal about precision. Classical reliability is a single number summarizing average precision across the entire sample tested — it obscures the fact that measurement accuracy varies by ability level. IRT's conditional standard error of measurement (CSEM) is a function: it shows a different SE at each theta value, revealing that examinees near the information peak are measured much more precisely than those at the tails. Two examinees who took the same test can have genuinely different measurement precision depending on where they fall on the theta scale. This conditional information is critical for fair high-stakes decisions and cannot be recovered from a classical reliability coefficient.

Question 5

Why does item information peak at the item's difficulty parameter θ = b, and what does this imply for how a computerized adaptive testing (CAT) algorithm selects items?

Model answer: Item information is proportional to the squared slope of the item characteristic curve at a given theta. The ICC slope is steepest at the difficulty parameter b — the inflection point of the logistic function — because this is where examinee ability differences produce the largest differences in response probability. Above and below b, the curve flattens, so ability differences produce smaller response probability differences and less information. For CAT, this implies the algorithm should select the item whose b parameter is closest to the current theta estimate, because that item maximally discriminates among examinees near the estimated ability level. After each response updates the theta estimate, the next item is again chosen to maximize information at the new estimate, producing a test that is always optimally targeted.

The practical consequence is profound: a well-functioning CAT exam can measure ability as precisely as a much longer fixed-format test, because every item contributes maximum information. A student at θ = 1.0 takes an exam where items rapidly converge on difficulties near 1.0, rather than wasting items that are too easy or too hard. The CSEM at the final theta estimate reflects how well-targeted the adaptive selection was — if items clustered near the true theta, the SE will be small.