  We propose to predict the future trajectories of observed agents (e.g.,
pedestrians or vehicles) by estimating and using their goals at multiple time
scales. We argue that the goal of a moving agent may change over time, and
modeling goals continuously provides more accurate and detailed information for
future trajectory estimation. To this end, we present a recurrent network for
trajectory prediction, called Stepwise Goal-Driven Network (SGNet). Unlike
prior work that models only a single, long-term goal, SGNet estimates and uses
goals at multiple temporal scales. In particular, it incorporates an encoder
that captures historical information, a stepwise goal estimator that predicts
successive goals into the future, and a decoder that predicts future
trajectory. We evaluate our model on three first-person traffic datasets
(HEV-I, JAAD, and PIE) as well as on three bird's eye view datasets (NuScenes,
ETH, and UCY), and show that our model achieves state-of-the-art results on all
datasets. Code has been made available at:
https://github.com/ChuhuaW/SGNet.pytorch.
