  We present a novel deep neural network architecture for end-to-end scene flow
estimation that directly operates on large-scale 3D point clouds. Inspired by
Bilateral Convolutional Layers (BCL), we propose novel DownBCL, UpBCL, and
CorrBCL operations that restore structural information from unstructured point
clouds, and fuse information from two consecutive point clouds. Operating on
discrete and sparse permutohedral lattice points, our architectural design is
parsimonious in computational cost. Our model can efficiently process a pair of
point cloud frames at once with a maximum of 86K points per frame. Our approach
achieves state-of-the-art performance on the FlyingThings3D and KITTI Scene
Flow 2015 datasets. Moreover, trained on synthetic data, our approach shows
great generalization ability on real-world data and on different point
densities without fine-tuning.
