  Robust motion forecasting of the dynamic scene is a critical component of an
autonomous vehicle. It is a challenging problem due to the heterogeneity in the
scene and the inherent uncertainties in the problem. To improve the accuracy of
motion forecasting, in this work, we identify a new consistency constraint in
this task, that is an agent's future trajectory should be coherent with its
history observations and visa versa. To leverage this property, we propose a
novel cycle consistency training scheme and define a novel cycle loss to
encourage this consistency. In particular, we reverse the predicted future
trajectory backward in time and feed it back into the prediction model to
predict the history and compute the loss as an additional cycle loss term.
Through our experiments on the Argoverse dataset, we demonstrate that cycle
loss can improve the performance of competitive motion forecasting models.
