  Behavior prediction models have proliferated in recent years, especially in
the popular real-world robotics application of autonomous driving, where
representing the distribution over possible futures of moving agents is
essential for safe and comfortable motion planning. In these models, the choice
of coordinate frames to represent inputs and outputs has crucial trade offs
which broadly fall into one of two categories. Agent-centric models transform
inputs and perform inference in agent-centric coordinates. These models are
intrinsically invariant to translation and rotation between scene elements, are
best-performing on public leaderboards, but scale quadratically with the number
of agents and scene elements. Scene-centric models use a fixed coordinate
system to process all agents. This gives them the advantage of sharing
representations among all agents, offering efficient amortized inference
computation which scales linearly with the number of agents. However, these
models have to learn invariance to translation and rotation between scene
elements, and typically underperform agent-centric models. In this work, we
develop knowledge distillation techniques between probabilistic motion
forecasting models, and apply these techniques to close the gap in performance
between agent-centric and scene-centric models. This improves scene-centric
model performance by 13.2% on the public Argoverse benchmark, 7.8% on Waymo
Open Dataset and up to 9.4% on a large In-House dataset. These improved
scene-centric models rank highly in public leaderboards and are up to 15 times
more efficient than their agent-centric teacher counterparts in busy scenes.
