Questions: Distractor Analysis and Multiple-Choice Item Evaluation

Question 1

An item analysis shows that distractor option B is selected by 38% of high-scoring examinees and only 12% of low-scoring examinees. What does this pattern most likely indicate?

• A. Option B is a highly effective distractor because many examinees selected it
• B. Option B may be defensible or ambiguous, and the scoring key should be reviewed
• C. The item is too difficult and should be removed from the test
• D. Low-scoring examinees are not reading carefully enough

Answer: (B)

A functioning distractor should have a negative correlation with test score — low scorers choose it more, high scorers avoid it. When high-scorers choose a distractor more than low-scorers, the option-biserial correlation is positive, which is a red flag. High-scorers are presumably more knowledgeable, so if they are selecting a 'wrong' answer, it may not actually be wrong — the option may be ambiguous, technically defensible, or reveal a scoring key error. This is precisely when item review is needed.

Question 2

What property must a distractor have to be considered 'functioning' in a well-designed multiple-choice item?

• A. It must be selected by at least 5% of examinees at all ability levels
• B. It must be selected more frequently by lower-scoring examinees than by higher-scoring examinees
• C. It must closely resemble the correct answer in surface form to maximize difficulty
• D. It must be selected by high-scoring examinees to confirm they carefully considered it

Answer: (B)

The diagnostic signature of a functioning distractor is a negative option-level point-biserial correlation — low-scorers select it more often than high-scorers. This mirrors item discrimination logic: the distractor successfully attracts examinees who hold a specific misconception or gap, while well-prepared examinees correctly reject it. A distractor that is equally likely to attract high and low scorers provides no diagnostic information and may signal item flaws.

Question 3

An unselected distractor — one chosen by almost no examinees — is a problem in multiple-choice item development because it wastes an option slot that could carry diagnostic information.

• T. True
• F. False

Answer: True

Distractors serve a purpose: they should attract examinees who hold a specific, predictable misconception and thereby reveal diagnostic information about what examinees do and don't understand. A distractor that nobody selects contributes nothing — it is not attracting the misconception it was presumably written to target. That option slot could instead represent a more commonly held error, making the item more informative and the test more valid.

Question 4

A distractor is functioning well if high-scoring examinees avoid it, even if low-scoring examinees also rarely select it.

• T. True
• F. False

Answer: False

A distractor requires two things to be functioning: high scorers should avoid it AND low scorers should select it. If both groups avoid it, the distractor is simply unselected — it represents no real misconception in the test population. The negative correlation between option selection and test score is the defining characteristic of a functioning distractor. High scorers avoiding it alone does not make it functional; the distractor must also attract the less-prepared examinees.

Question 5

What does it mean for a distractor to be 'diagnostic,' and why is this property important in test development?

Model answer: A diagnostic distractor represents a specific, predictable misconception or error pattern — when an examinee chooses it, this reveals something meaningful about their knowledge gap. A distractor is diagnostic when examinees who select it share a common error: a procedural mistake, a conceptual confusion, or a misremembered fact. This matters because it transforms the multiple-choice item from a simple right/wrong indicator into a tool that reveals the nature of examinee understanding, enabling targeted instructional follow-up.

Non-diagnostic distractors — those that attract responses at random or that attract nobody — provide only noise. The goal of distractor construction is to represent the actual error space of the examinee population, so that wrong answers carry as much information as right answers. In formative assessment settings, this diagnostic function is especially valuable.