  One of the core components in online multiple object tracking (MOT)
frameworks is associating new detections with existing tracklets, typically
done via a scoring function. Despite the great advances in MOT, designing a
reliable scoring function remains a challenge. In this paper, we introduce a
probabilistic autoregressive generative model to score tracklet proposals by
directly measuring the likelihood that a tracklet represents natural motion.
One key property of our model is its ability to generate multiple likely
futures of a tracklet given partial observations. This allows us to not only
score tracklets but also effectively maintain existing tracklets when the
detector fails to detect some objects even for a long time, e.g., due to
occlusion, by sampling trajectories so as to inpaint the gaps caused by
misdetection. Our experiments demonstrate the effectiveness of our approach to
scoring and inpainting tracklets on several MOT benchmark datasets. We
additionally show the generality of our generative model by using it to produce
future representations in the challenging task of human motion prediction.
