  We present an approach for detecting and estimating the 3D poses of objects
in images that requires only an untextured CAD model and no training phase for
new objects. Our approach combines Deep Learning and 3D geometry: It relies on
an embedding of local 3D geometry to match the CAD models to the input images.
For points at the surface of objects, this embedding can be computed directly
from the CAD model; for image locations, we learn to predict it from the image
itself. This establishes correspondences between 3D points on the CAD model and
2D locations of the input images. However, many of these correspondences are
ambiguous as many points may have similar local geometries. We show that we can
use Mask-RCNN in a class-agnostic way to detect the new objects without
retraining and thus drastically limit the number of possible correspondences.
We can then robustly estimate a 3D pose from these discriminative
correspondences using a RANSAC- like algorithm. We demonstrate the performance
of this approach on the T-LESS dataset, by using a small number of objects to
learn the embedding and testing it on the other objects. Our experiments show
that our method is on par or better than previous methods.
