  Identity Switching remains one of the main difficulties Multiple Object
Tracking (MOT) algorithms have to deal with. Many state-of-the-art approaches
now use sequence models to solve this problem but their training can be
affected by biases that decrease their efficiency. In this paper, we introduce
a new training procedure that confronts the algorithm to its own mistakes while
explicitly attempting to minimize the number of switches, which results in
better training. We propose an iterative scheme of building a rich training set
and using it to learn a scoring function that is an explicit proxy for the
target tracking metric. Whether using only simple geometric features or more
sophisticated ones that also take appearance into account, our approach
outperforms the state-of-the-art on several MOT benchmarks.
