  The complementary benefits from visible and thermal infrared data are widely
utilized in various computer vision task, such as visual tracking, semantic
segmentation and object detection, but rarely explored in Multiple Object
Tracking (MOT). In this work, we contribute a large-scale Visible-Thermal video
benchmark for MOT, called VT-MOT. VT-MOT has the following main advantages. 1)
The data is large scale and high diversity. VT-MOT includes 582 video sequence
pairs, 401k frame pairs from surveillance, drone, and handheld platforms. 2)
The cross-modal alignment is highly accurate. We invite several professionals
to perform both spatial and temporal alignment frame by frame. 3) The
annotation is dense and high-quality. VT-MOT has 3.99 million annotation boxes
annotated and double-checked by professionals, including heavy occlusion and
object re-acquisition (object disappear and reappear) challenges. To provide a
strong baseline, we design a simple yet effective tracking framework, which
effectively fuses temporal information and complementary information of two
modalities in a progressive manner, for robust visible-thermal MOT. A
comprehensive experiment are conducted on VT-MOT and the results prove the
superiority and effectiveness of the proposed method compared with
state-of-the-art methods. From the evaluation results and analysis, we specify
several potential future directions for visible-thermal MOT. The project is
released in https://github.com/wqw123wqw/PFTrack.
