  3D multi-object tracking in LiDAR point clouds is a key ingredient for
self-driving vehicles. Existing methods are predominantly based on the
tracking-by-detection pipeline and inevitably require a heuristic matching step
for the detection association. In this paper, we present SimTrack to simplify
the hand-crafted tracking paradigm by proposing an end-to-end trainable model
for joint detection and tracking from raw point clouds. Our key design is to
predict the first-appear location of each object in a given snippet to get the
tracking identity and then update the location based on motion estimation. In
the inference, the heuristic matching step can be completely waived by a simple
read-off operation. SimTrack integrates the tracked object association, newborn
object detection, and dead track killing in a single unified model. We conduct
extensive evaluations on two large-scale datasets: nuScenes and Waymo Open
Dataset. Experimental results reveal that our simple approach compares
favorably with the state-of-the-art methods while ruling out the heuristic
matching rules.
