  3D multi-object tracking and trajectory prediction are two crucial modules in
autonomous driving systems. Generally, the two tasks are handled separately in
traditional paradigms and a few methods have started to explore modeling these
two tasks in a joint manner recently. However, these approaches suffer from the
limitations of single-frame training and inconsistent coordinate
representations between tracking and prediction tasks. In this paper, we
propose a streaming and unified framework for joint 3D Multi-Object Tracking
and trajectory Prediction (StreamMOTP) to address the above challenges.
Firstly, we construct the model in a streaming manner and exploit a memory bank
to preserve and leverage the long-term latent features for tracked objects more
effectively. Secondly, a relative spatio-temporal positional encoding strategy
is introduced to bridge the gap of coordinate representations between the two
tasks and maintain the pose-invariance for trajectory prediction. Thirdly, we
further improve the quality and consistency of predicted trajectories with a
dual-stream predictor. We conduct extensive experiments on popular nuSences
dataset and the experimental results demonstrate the effectiveness and
superiority of StreamMOTP, which outperforms previous methods significantly on
both tasks. Furthermore, we also prove that the proposed framework has great
potential and advantages in actual applications of autonomous driving.
