  Lidar point cloud distortion from moving object is an important problem in
autonomous driving, and recently becomes even more demanding with the emerging
of newer lidars, which feature back-and-forth scanning patterns. Accurately
estimating moving object velocity would not only provide a tracking capability
but also correct the point cloud distortion with more accurate description of
the moving object. Since lidar measures the time-of-flight distance but with a
sparse angular resolution, the measurement is precise in the radial measurement
but lacks angularly. Camera on the other hand provides a dense angular
resolution. In this paper, Gaussian-based lidar and camera fusion is proposed
to estimate the full velocity and correct the lidar distortion. A probabilistic
Kalman-filter framework is provided to track the moving objects, estimate their
velocities and simultaneously correct the point clouds distortions. The
framework is evaluated on real road data and the fusion method outperforms the
traditional ICP-based and point-cloud only method. The complete working
framework is open-sourced
(https://github.com/ISEE-Technology/lidar-with-velocity) to accelerate the
adoption of the emerging lidars.
