  Object tracking and 3D reconstruction are often performed together, with
tracking used as input for reconstruction. However, the obtained
reconstructions also provide useful information for improving tracking. We
propose a novel method that closes this loop, first tracking to reconstruct,
and then reconstructing to track. Our approach, MOTSFusion (Multi-Object
Tracking, Segmentation and dynamic object Fusion), exploits the 3D motion
extracted from dynamic object reconstructions to track objects through long
periods of complete occlusion and to recover missing detections. Our approach
first builds up short tracklets using 2D optical flow, and then fuses these
into dynamic 3D object reconstructions. The precise 3D object motion of these
reconstructions is used to merge tracklets through occlusion into long-term
tracks, and to locate objects when detections are missing. On KITTI, our
reconstruction-based tracking reduces the number of ID switches of the initial
tracklets by more than 50%, and outperforms all previous approaches for both
bounding box and segmentation tracking.
