  3D multi-object tracking (MOT) is vital for many applications including
autonomous driving vehicles and service robots. With the commonly used
tracking-by-detection paradigm, 3D MOT has made important progress in recent
years. However, these methods only use the detection boxes of the current frame
to obtain trajectory-box association results, which makes it impossible for the
tracker to recover objects missed by the detector. In this paper, we present
TrajectoryFormer, a novel point-cloud-based 3D MOT framework. To recover the
missed object by detector, we generates multiple trajectory hypotheses with
hybrid candidate boxes, including temporally predicted boxes and current-frame
detection boxes, for trajectory-box association. The predicted boxes can
propagate object's history trajectory information to the current frame and thus
the network can tolerate short-term miss detection of the tracked objects. We
combine long-term object motion feature and short-term object appearance
feature to create per-hypothesis feature embedding, which reduces the
computational overhead for spatial-temporal encoding. Additionally, we
introduce a Global-Local Interaction Module to conduct information interaction
among all hypotheses and models their spatial relations, leading to accurate
estimation of hypotheses. Our TrajectoryFormer achieves state-of-the-art
performance on the Waymo 3D MOT benchmarks. Code is available at
https://github.com/poodarchu/EFG .
