  Iterative Closest Point (ICP) solves the rigid point cloud registration
problem iteratively in two steps: (1) make hard assignments of spatially
closest point correspondences, and then (2) find the least-squares rigid
transformation. The hard assignments of closest point correspondences based on
spatial distances are sensitive to the initial rigid transformation and
noisy/outlier points, which often cause ICP to converge to wrong local minima.
In this paper, we propose the RPM-Net -- a less sensitive to initialization and
more robust deep learning-based approach for rigid point cloud registration. To
this end, our network uses the differentiable Sinkhorn layer and annealing to
get soft assignments of point correspondences from hybrid features learned from
both spatial coordinates and local geometry. To further improve registration
performance, we introduce a secondary network to predict optimal annealing
parameters. Unlike some existing methods, our RPM-Net handles missing
correspondences and point clouds with partial visibility. Experimental results
show that our RPM-Net achieves state-of-the-art performance compared to
existing non-deep learning and recent deep learning methods. Our source code is
available at the project website https://github.com/yewzijian/RPMNet .
