  In this paper we propose a novel approach to estimate dense optical flow from
sparse lidar data acquired on an autonomous vehicle. This is intended to be
used as a drop-in replacement of any image-based optical flow system when
images are not reliable due to e.g. adverse weather conditions or at night. In
order to infer high resolution 2D flows from discrete range data we devise a
three-block architecture of multiscale filters that combines multiple
intermediate objectives, both in the lidar and image domain. To train this
network we introduce a dataset with approximately 20K lidar samples of the
Kitti dataset which we have augmented with a pseudo ground-truth image-based
optical flow computed using FlowNet2. We demonstrate the effectiveness of our
approach on Kitti, and show that despite using the low-resolution and sparse
measurements of the lidar, we can regress dense optical flow maps which are at
par with those estimated with image-based methods.
