  Modern autonomous systems rely on perception modules to process complex
sensor measurements into state estimates. These estimates are then passed to a
controller, which uses them to make safety-critical decisions. It is therefore
important that we design perception systems to minimize errors that reduce the
overall safety of the system. We develop a risk-driven approach to designing
perception systems that accounts for the effect of perceptual errors on the
performance of the fully-integrated, closed-loop system. We formulate a risk
function to quantify the effect of a given perceptual error on overall safety,
and show how we can use it to design safer perception systems by including a
risk-dependent term in the loss function and generating training data in
risk-sensitive regions. We evaluate our techniques on a realistic vision-based
aircraft detect and avoid application and show that risk-driven design reduces
collision risk by 37% over a baseline system.
