  Temporal modeling of objects is a key challenge in multiple object tracking
(MOT). Existing methods track by associating detections through motion-based
and appearance-based similarity heuristics. The post-processing nature of
association prevents end-to-end exploitation of temporal variations in video
sequence. In this paper, we propose MOTR, which extends DETR and introduces
track query to model the tracked instances in the entire video. Track query is
transferred and updated frame-by-frame to perform iterative prediction over
time. We propose tracklet-aware label assignment to train track queries and
newborn object queries. We further propose temporal aggregation network and
collective average loss to enhance temporal relation modeling. Experimental
results on DanceTrack show that MOTR significantly outperforms state-of-the-art
method, ByteTrack by 6.5% on HOTA metric. On MOT17, MOTR outperforms our
concurrent works, TrackFormer and TransTrack, on association performance. MOTR
can serve as a stronger baseline for future research on temporal modeling and
Transformer-based trackers. Code is available at
https://github.com/megvii-research/MOTR.
