Questions: Safety Verification and Validation for Autonomous Systems
1 questions to test your understanding
Score: 0 / 1
Question 1 Multiple Choice
A perception system detects a pedestrian with 95% confidence. In traditional software engineering, this would be deemed 'highly reliable.' Why is this insufficient for autonomous driving safety, and what additional factors must be considered?
AThe detection is reliable; 95% is sufficient for any application
B95% accuracy on test data does not guarantee 95% accuracy on all future data (distribution shift); moreover, the failure mode (missing a pedestrian = collision) is safety-critical. We must measure false negative rate (pedestrians missed), test on diverse populations, ensure rare scenarios (e.g., small child partially occluded) are not missed, and establish what detection rate is required to achieve target safety metrics (e.g., fewer than 1 collision per 100 million miles)
CPerception is unimportant for safety; only control safety matters
DDetection confidence values are always accurate, so we can trust them directly
ODD is a key concept in ISO 26262 and emerging AV safety standards. It prevents manufacturers from overselling capabilities while allowing systems to be deployed progressively as their operational domain expands. A vehicle that is safe on highways with good weather can be deployed there; extending it to rain, night, or residential streets requires additional validation. The ODD represents the honesty principle in safety: autonomous systems have limits; those limits must be explicit.