  Human trajectory prediction has received increased attention lately due to
its importance in applications such as autonomous vehicles and indoor robots.
However, most existing methods make predictions based on human-labeled
trajectories and ignore the errors and noises in detection and tracking. In
this paper, we study the problem of human trajectory forecasting in raw videos,
and show that the prediction accuracy can be severely affected by various types
of tracking errors. Accordingly, we propose a simple yet effective strategy to
correct the tracking failures by enforcing prediction consistency over time.
The proposed "re-tracking" algorithm can be applied to any existing tracking
and prediction pipelines. Experiments on public benchmark datasets demonstrate
that the proposed method can improve both tracking and prediction performance
in challenging real-world scenarios. The code and data are available at
https://git.io/retracking-prediction.
