  The recent trend in vision-based multi-object tracking (MOT) is heading
towards leveraging the representational power of deep learning to jointly learn
to detect and track objects. However, existing methods train only certain
sub-modules using loss functions that often do not correlate with established
tracking evaluation measures such as Multi-Object Tracking Accuracy (MOTA) and
Precision (MOTP). As these measures are not differentiable, the choice of
appropriate loss functions for end-to-end training of multi-object tracking
methods is still an open research problem. In this paper, we bridge this gap by
proposing a differentiable proxy of MOTA and MOTP, which we combine in a loss
function suitable for end-to-end training of deep multi-object trackers. As a
key ingredient, we propose a Deep Hungarian Net (DHN) module that approximates
the Hungarian matching algorithm. DHN allows estimating the correspondence
between object tracks and ground truth objects to compute differentiable
proxies of MOTA and MOTP, which are in turn used to optimize deep trackers
directly. We experimentally demonstrate that the proposed differentiable
framework improves the performance of existing multi-object trackers, and we
establish a new state of the art on the MOTChallenge benchmark. Our code is
publicly available from https://github.com/yihongXU/deepMOT.
