  Estimating the states of surrounding traffic participants stays at the core
of autonomous driving. In this paper, we study a novel setting of this problem:
model-free single-object tracking (SOT), which takes the object state in the
first frame as input, and jointly solves state estimation and tracking in
subsequent frames. The main purpose for this new setting is to break the strong
limitation of the popular "detection and tracking" scheme in multi-object
tracking. Moreover, we notice that shape completion by overlaying the point
clouds, which is a by-product of our proposed task, not only improves the
performance of state estimation but also has numerous applications. As no
benchmark for this task is available so far, we construct a new dataset
LiDAR-SOT and corresponding evaluation protocols based on the Waymo Open
dataset. We then propose an optimization-based algorithm called SOTracker
involving point cloud registration, vehicle shapes, correspondence, and motion
priors. Our quantitative and qualitative results prove the effectiveness of our
SOTracker and reveal the challenging cases for SOT in point clouds, including
the sparsity of LiDAR data, abrupt motion variation, etc. Finally, we also
explore how the proposed task and algorithm may benefit other autonomous
driving applications, including simulating LiDAR scans, generating motion data,
and annotating optical flow. The code and protocols for our benchmark and
algorithm are available at https://github.com/TuSimple/LiDAR_SOT/. A video
demonstration is at https://www.youtube.com/watch?v=BpHixKs91i8.
