  We present a novel transformer-based architecture for global multi-object
tracking. Our network takes a short sequence of frames as input and produces
global trajectories for all objects. The core component is a global tracking
transformer that operates on objects from all frames in the sequence. The
transformer encodes object features from all frames, and uses trajectory
queries to group them into trajectories. The trajectory queries are object
features from a single frame and naturally produce unique trajectories. Our
global tracking transformer does not require intermediate pairwise grouping or
combinatorial association, and can be jointly trained with an object detector.
It achieves competitive performance on the popular MOT17 benchmark, with 75.3
MOTA and 59.1 HOTA. More importantly, our framework seamlessly integrates into
state-of-the-art large-vocabulary detectors to track any objects. Experiments
on the challenging TAO dataset show that our framework consistently improves
upon baselines that are based on pairwise association, outperforming published
works by a significant 7.7 tracking mAP. Code is available at
https://github.com/xingyizhou/GTR.
