  We present a method to integrate real-time out-of-distribution (OOD)
detection for neural network trajectory predictors, and to adapt the control
strategy of a robot (e.g., a self-driving car or drone) to preserve safety
while operating in OOD regimes. Specifically, we use a neural network ensemble
to predict the trajectory for a dynamic obstacle (such as a pedestrian), and
use the maximum singular value of the empirical covariance among the ensemble
as a signal for OOD detection. We calibrate this signal with a small fraction
of held-out training data using the methodology of conformal prediction, to
derive an OOD detector with probabilistic guarantees on the false-positive rate
of the detector, given a user-specified confidence level. During
in-distribution operation, we use an MPC controller to avoid collisions with
the obstacle based on the trajectory predicted by the neural network ensemble.
When OOD conditions are detected, we switch to a reachability-based controller
to guarantee safety under the worst-case actions of the obstacle. We verify our
method in extensive autonomous driving simulations in a pedestrian crossing
scenario, showing that our OOD detector obtains the desired accuracy rate
within a theoretically-predicted range. We also demonstrate the effectiveness
of our method with real pedestrian data. We show improved safety and less
conservatism in comparison with two state-of-the-art methods that also use
conformal prediction, but without OOD adaptation.
