  Existing learning-based methods for object pose estimation in RGB images are
mostly model-specific or category based. They lack the capability to generalize
to new object categories at test time, hence severely hindering their
practicability and scalability. Notably, recent attempts have been made to
solve this issue, but they still require accurate 3D data of the object surface
at both train and test time. In this paper, we introduce a novel approach that
can estimate in a single forward pass the pose of objects never seen during
training, given minimum input. In contrast to existing state-of-the-art
approaches, which rely on task-specific modules, our proposed model is entirely
based on a transformer architecture, which can benefit from recently proposed
3D-geometry general pretraining. We conduct extensive experiments and report
state-of-the-art one-shot performance on the challenging LINEMOD benchmark.
Finally, extensive ablations allow us to determine good practices with this
relatively new type of architecture in the field.
