  Motion prediction is a critical part of self-driving technology, responsible
for inferring future behavior of traffic actors in autonomous vehicle's
surroundings. In order to ensure safe and efficient operations, prediction
models need to output accurate trajectories that obey the map constraints. In
this paper, we address this task and propose a novel ellipse loss that allows
the models to better reason about scene compliance and predict more realistic
trajectories. Ellipse loss penalizes off-road predictions directly in a
supervised manner, by projecting the output trajectories into the top-down map
frame using a differentiable trajectory rasterizer module. Moreover, it takes
into account actor dimensions and orientation, providing more direct training
signals to the model. We applied ellipse loss to a recently proposed
state-of-the-art joint detection-prediction model to showcase its benefits.
Evaluation on large-scale autonomous driving data strongly indicates that the
method allows for more accurate and more realistic trajectory predictions.
