  Accurate and reliable tracking of multiple moving objects in 3D space is an
essential component of urban scene understanding. This is a challenging task
because it requires the assignment of detections in the current frame to the
predicted objects from the previous one. Existing filter-based approaches tend
to struggle if this initial assignment is not correct, which can happen easily.
We propose a novel optimization-based approach that does not rely on explicit
and fixed assignments. Instead, we represent the result of an off-the-shelf 3D
object detector as Gaussian mixture model, which is incorporated in a factor
graph framework. This gives us the flexibility to assign all detections to all
objects simultaneously. As a result, the assignment problem is solved
implicitly and jointly with the 3D spatial multi-object state estimation using
non-linear least squares optimization. Despite its simplicity, the proposed
algorithm achieves robust and reliable tracking results and can be applied for
offline as well as online tracking. We demonstrate its performance on the real
world KITTI tracking dataset and achieve better results than many
state-of-the-art algorithms. Especially the consistency of the estimated tracks
is superior offline as well as online.
