  Traditionally multi-object tracking and object detection are performed using
separate systems with most prior works focusing exclusively on one of these
aspects over the other. Tracking systems clearly benefit from having access to
accurate detections, however and there is ample evidence in literature that
detectors can benefit from tracking which, for example, can help to smooth
predictions over time. In this paper we focus on the tracking-by-detection
paradigm for autonomous driving where both tasks are mission critical. We
propose a conceptually simple and efficient joint model of detection and
tracking, called RetinaTrack, which modifies the popular single stage RetinaNet
approach such that it is amenable to instance-level embedding training. We
show, via evaluations on the Waymo Open Dataset, that we outperform a recent
state of the art tracking algorithm while requiring significantly less
computation. We believe that our simple yet effective approach can serve as a
strong baseline for future work in this area.
