  We propose a method for estimating the 6DoF pose of a rigid object with an
available 3D model from a single RGB image. Unlike classical
correspondence-based methods which predict 3D object coordinates at pixels of
the input image, the proposed method predicts 3D object coordinates at 3D query
points sampled in the camera frustum. The move from pixels to 3D points, which
is inspired by recent PIFu-style methods for 3D reconstruction, enables
reasoning about the whole object, including its (self-)occluded parts. For a 3D
query point associated with a pixel-aligned image feature, we train a
fully-connected neural network to predict: (i) the corresponding 3D object
coordinates, and (ii) the signed distance to the object surface, with the first
defined only for query points in the surface vicinity. We call the mapping
realized by this network as Neural Correspondence Field. The object pose is
then robustly estimated from the predicted 3D-3D correspondences by the
Kabsch-RANSAC algorithm. The proposed method achieves state-of-the-art results
on three BOP datasets and is shown superior especially in challenging cases
with occlusion. The project website is at: linhuang17.github.io/NCF.
