  We introduce a motion forecasting (behavior prediction) method that meets the
latency requirements for autonomous driving in dense urban environments without
sacrificing accuracy. A whole-scene sparse input representation allows StopNet
to scale to predicting trajectories for hundreds of road agents with reliable
latency. In addition to predicting trajectories, our scene encoder lends itself
to predicting whole-scene probabilistic occupancy grids, a complementary output
representation suitable for busy urban environments. Occupancy grids allow the
AV to reason collectively about the behavior of groups of agents without
processing their individual trajectories. We demonstrate the effectiveness of
our sparse input representation and our model in terms of computation and
accuracy over three datasets. We further show that co-training consistent
trajectory and occupancy predictions improves upon state-of-the-art performance
under standard metrics.
