  Predicting the trajectories of road agents is essential for autonomous
driving systems. The recent mainstream methods follow a static paradigm, which
predicts the future trajectory by using a fixed duration of historical frames.
These methods make the predictions independently even at adjacent time steps,
which leads to potential instability and temporal inconsistency. As successive
time steps have largely overlapping historical frames, their forecasting should
have intrinsic correlation, such as overlapping predicted trajectories should
be consistent, or be different but share the same motion goal depending on the
road situation. Motivated by this, in this work, we introduce HPNet, a novel
dynamic trajectory forecasting method. Aiming for stable and accurate
trajectory forecasting, our method leverages not only historical frames
including maps and agent states, but also historical predictions. Specifically,
we newly design a Historical Prediction Attention module to automatically
encode the dynamic relationship between successive predictions. Besides, it
also extends the attention range beyond the currently visible window
benefitting from the use of historical predictions. The proposed Historical
Prediction Attention together with the Agent Attention and Mode Attention is
further formulated as the Triple Factorized Attention module, serving as the
core design of HPNet.Experiments on the Argoverse and INTERACTION datasets show
that HPNet achieves state-of-the-art performance, and generates accurate and
stable future trajectories. Our code are available at
https://github.com/XiaolongTang23/HPNet.
