Questions: Surveillance System Performance Metrics

Question 1

A state health department lowers its reporting threshold for a rare infectious disease, capturing 30% more true cases. An epidemiologist warns this change will create resource problems. What is her most likely concern?

• A. Lower thresholds slow processing speed because more paperwork must be filed for each case
• B. Increased sensitivity at very low disease prevalence reduces positive predictive value, generating more false-positive reports that must be investigated and that consume finite public health resources
• C. Sensitivity and specificity are independent of the reporting threshold, so changing the threshold cannot affect the false-positive rate
• D. Lower thresholds improve sensitivity and specificity simultaneously, so there is no trade-off to worry about

Answer: (B)

This is the PPV problem applied to surveillance. When a disease is rare, even a modestly imperfect system generates many false positives relative to true positives. Lowering the reporting threshold captures more true cases (sensitivity rises) but also accepts more false-positive reports. Each false-positive triggers contact tracing, environmental investigation, or patient follow-up — real costs that grow as the false-positive rate increases. The right threshold depends on disease severity: a highly transmissible pathogen justifies the resource cost; a less urgent condition may not.

Question 2

A public health team wants to improve a salmonellosis surveillance system that detects only 1 in 29 actual cases. Which intervention strategy is most directly targeted at finding the true bottleneck?

• A. Hire additional staff at the health department to process case reports more quickly
• B. Lower the case definition threshold to include milder presentations
• C. Systematically audit each stage of the reporting pipeline — from care-seeking to testing to reporting — to identify which step loses the most cases, then intervene there
• D. Increase laboratory capacity to run more diagnostic tests per day

Answer: (C)

Bottleneck analysis is the key concept here. A surveillance pipeline with 3.5% sensitivity is losing cases at many points: some sick people never seek care, some who seek care are never tested, some who test positive are never reported. Intervening at the wrong stage (e.g., improving laboratory capacity when most cases are lost before patients see a doctor) wastes resources with minimal gain. Mapping the pipeline stage-by-stage and quantifying loss at each step is the only way to identify the correct intervention point.

Question 3

A surveillance system can achieve high sensitivity for detecting true cases yet still have low positive predictive value (PPV) when the disease being surveilled is very rare in the population.

• T. True
• F. False

Answer: True

This is the same PPV paradox you learned in biostatistics, operating at the population level. PPV depends on three things: sensitivity, specificity, and prevalence. When prevalence is very low, even a system with 99% specificity will generate many false positives for every true positive, because there are so many more non-cases in the population. A surveillance system detecting a disease affecting 1 in 100,000 people will be overwhelmed with false alarms even if its specificity is high. This is why PPV — not just sensitivity and specificity — must be considered when evaluating surveillance system performance.

Question 4

Timeliness and sensitivity measure the same underlying property of a surveillance system — a system with higher sensitivity will automatically detect cases more quickly.

• T. True
• F. False

Answer: False

Timeliness and sensitivity are orthogonal dimensions. Sensitivity measures the proportion of true cases that are ever detected; timeliness measures the lag between when an event occurs and when the system reports it. A system could detect 80% of cases but take 3 weeks to report each one (high sensitivity, poor timeliness) — useless for real-time outbreak control. Conversely, a system might report immediately but only capture 20% of cases. Improving timeliness requires reducing delays in the reporting pipeline; improving sensitivity requires capturing more cases at each stage.

Question 5

Why does the appropriate trade-off between sensitivity and specificity in a surveillance system depend on the severity and transmissibility of the disease being surveilled?

Model answer: For a highly lethal or rapidly transmissible disease, missing a true case carries extreme consequences — an undetected smallpox case or early cluster could be catastrophic. The cost of a missed case vastly exceeds the cost of investigating a false positive, so it makes sense to sacrifice specificity for sensitivity. For a less severe or less transmissible condition, the calculus reverses: false-positive reports consume investigative resources and may harm individuals (stigma, unnecessary treatment), while missed cases carry lower population-level consequences. The correct balance is a policy judgment that requires knowing the stakes on both sides.

This is analogous to medical testing: for diseases where missing a diagnosis is lethal (e.g., certain cancers), clinicians use highly sensitive screening tests and accept more false positives that trigger confirmatory workup. For conditions where overdiagnosis causes substantial harm, more specific tests are preferred. Surveillance faces the same trade-off at population scale — the 'cost' of a false positive is investigative resources and potential stigma; the 'cost' of a false negative is undetected disease spread or burden.