Questions: Perception Pipeline for Autonomous Systems

Question 1

A camera-based object detector trained on daytime images with high lighting quality is deployed on an autonomous vehicle and fails catastrophically on first run during dawn with backlighting and shadows. Why does this occur, and how would you address it?

• A. The detector's neural network is too simple; it needs more layers
• B. The detector was trained on data with different lighting conditions than deployment; it lacks generalization to appearance variations. Address this by training on diverse lighting conditions, using data augmentation (simulating shadows and glare), or deploying multi-sensor fusion so camera failures don't disable the system
• C. Camera detectors are fundamentally unreliable; replace the camera with lidar
• D. The detector needs real-time processing; the failure is a computational speed issue

Answer: (B)

This design choice directly addresses the reliability requirements of autonomous systems. Single-sensor designs require that sensor to work in all conditions; multi-sensor systems can tolerate individual sensor failures. This is why the autonomous vehicle industry converged on camera + lidar + radar as the standard sensor suite.