  We introduce MegaPose, a method to estimate the 6D pose of novel objects,
that is, objects unseen during training. At inference time, the method only
assumes knowledge of (i) a region of interest displaying the object in the
image and (ii) a CAD model of the observed object. The contributions of this
work are threefold. First, we present a 6D pose refiner based on a
render&compare strategy which can be applied to novel objects. The shape and
coordinate system of the novel object are provided as inputs to the network by
rendering multiple synthetic views of the object's CAD model. Second, we
introduce a novel approach for coarse pose estimation which leverages a network
trained to classify whether the pose error between a synthetic rendering and an
observed image of the same object can be corrected by the refiner. Third, we
introduce a large-scale synthetic dataset of photorealistic images of thousands
of objects with diverse visual and shape properties and show that this
diversity is crucial to obtain good generalization performance on novel
objects. We train our approach on this large synthetic dataset and apply it
without retraining to hundreds of novel objects in real images from several
pose estimation benchmarks. Our approach achieves state-of-the-art performance
on the ModelNet and YCB-Video datasets. An extensive evaluation on the 7 core
datasets of the BOP challenge demonstrates that our approach achieves
performance competitive with existing approaches that require access to the
target objects during training. Code, dataset and trained models are available
on the project page: https://megapose6d.github.io/.
