  We propose Occupancy Flow Fields, a new representation for motion forecasting
of multiple agents, an important task in autonomous driving. Our representation
is a spatio-temporal grid with each grid cell containing both the probability
of the cell being occupied by any agent, and a two-dimensional flow vector
representing the direction and magnitude of the motion in that cell. Our method
successfully mitigates shortcomings of the two most commonly-used
representations for motion forecasting: trajectory sets and occupancy grids.
Although occupancy grids efficiently represent the probabilistic location of
many agents jointly, they do not capture agent motion and lose the agent
identities. To this end, we propose a deep learning architecture that generates
Occupancy Flow Fields with the help of a new flow trace loss that establishes
consistency between the occupancy and flow predictions. We demonstrate the
effectiveness of our approach using three metrics on occupancy prediction,
motion estimation, and agent ID recovery. In addition, we introduce the problem
of predicting speculative agents, which are currently-occluded agents that may
appear in the future through dis-occlusion or by entering the field of view. We
report experimental results on a large in-house autonomous driving dataset and
the public INTERACTION dataset, and show that our model outperforms
state-of-the-art models.
