  Tracking a crowd in 3D using multiple RGB cameras is a challenging task. Most
previous multi-camera tracking algorithms are designed for offline setting and
have high computational complexity. Robust real-time multi-camera 3D tracking
is still an unsolved problem. In this work, we propose a novel end-to-end
tracking pipeline, Deep Multi-Camera Tracking (DMCT), which achieves reliable
real-time multi-camera people tracking. Our DMCT consists of 1) a fast and
novel perspective-aware Deep GroudPoint Network, 2) a fusion procedure for
ground-plane occupancy heatmap estimation, 3) a novel Deep Glimpse Network for
person detection and 4) a fast and accurate online tracker. Our design fully
unleashes the power of deep neural network to estimate the "ground point" of
each person in each color image, which can be optimized to run efficiently and
robustly. Our fusion procedure, glimpse network and tracker merge the results
from different views, find people candidates using multiple video frames and
then track people on the fused heatmap. Our system achieves the
state-of-the-art tracking results while maintaining real-time performance.
Apart from evaluation on the challenging WILDTRACK dataset, we also collect two
more tracking datasets with high-quality labels from two different environments
and camera settings. Our experimental results confirm that our proposed
real-time pipeline gives superior results to previous approaches.
