  In this paper, we present LookOut, a novel autonomy system that perceives the
environment, predicts a diverse set of futures of how the scene might unroll
and estimates the trajectory of the SDV by optimizing a set of contingency
plans over these future realizations. In particular, we learn a diverse joint
distribution over multi-agent future trajectories in a traffic scene that
covers a wide range of future modes with high sample efficiency while
leveraging the expressive power of generative models. Unlike previous work in
diverse motion forecasting, our diversity objective explicitly rewards sampling
future scenarios that require distinct reactions from the self-driving vehicle
for improved safety. Our contingency planner then finds comfortable and
non-conservative trajectories that ensure safe reactions to a wide range of
future scenarios. Through extensive evaluations, we show that our model
demonstrates significantly more diverse and sample-efficient motion forecasting
in a large-scale self-driving dataset as well as safer and less-conservative
motion plans in long-term closed-loop simulations when compared to current
state-of-the-art models.
