  Deep neural network controllers for autonomous driving have recently
benefited from significant performance improvements, and have begun deployment
in the real world. Prior to their widespread adoption, safety guarantees are
needed on the controller behaviour that properly take account of the
uncertainty within the model as well as sensor noise. Bayesian neural networks,
which assume a prior over the weights, have been shown capable of producing
such uncertainty measures, but properties surrounding their safety have not yet
been quantified for use in autonomous driving scenarios. In this paper, we
develop a framework based on a state-of-the-art simulator for evaluating
end-to-end Bayesian controllers. In addition to computing pointwise uncertainty
measures that can be computed in real time and with statistical guarantees, we
also provide a method for estimating the probability that, given a scenario,
the controller keeps the car safe within a finite horizon. We experimentally
evaluate the quality of uncertainty computation by several Bayesian inference
methods in different scenarios and show how the uncertainty measures can be
combined and calibrated for use in collision avoidance. Our results suggest
that uncertainty estimates can greatly aid decision making in autonomous
driving.
