  Pose estimation of 3D objects in monocular images is a fundamental and
long-standing problem in computer vision. Existing deep learning approaches for
6D pose estimation typically rely on the assumption of availability of 3D
object models and 6D pose annotations. However, precise annotation of 6D poses
in real data is intricate, time-consuming and not scalable, while synthetic
data scales well but lacks realism. To avoid these problems, we present a
weakly-supervised reconstruction-based pipeline, named NeRF-Pose, which needs
only 2D object segmentation and known relative camera poses during training.
Following the first-reconstruct-then-regress idea, we first reconstruct the
objects from multiple views in the form of an implicit neural representation.
Then, we train a pose regression network to predict pixel-wise 2D-3D
correspondences between images and the reconstructed model. At inference, the
approach only needs a single image as input. A NeRF-enabled PnP+RANSAC
algorithm is used to estimate stable and accurate pose from the predicted
correspondences. Experiments on LineMod and LineMod-Occlusion show that the
proposed method has state-of-the-art accuracy in comparison to the best 6D pose
estimation methods in spite of being trained only with weak labels. Besides, we
extend the Homebrewed DB dataset with more real training images to support the
weakly supervised task and achieve compelling results on this dataset. The
extended dataset and code will be released soon.
