  Accurate 6D object pose estimation is an important task for a variety of
robotic applications such as grasping or localization. It is a challenging task
due to object symmetries, clutter and occlusion, but it becomes more
challenging when additional information, such as depth and 3D models, is not
provided. We present a transformer-based approach that takes an RGB image as
input and predicts a 6D pose for each object in the image. Besides the image,
our network does not require any additional information such as depth maps or
3D object models. First, the image is passed through an object detector to
generate feature maps and to detect objects. Then, the feature maps are fed
into a transformer with the detected bounding boxes as additional information.
Afterwards, the output object queries are processed by a separate translation
and rotation head. We achieve state-of-the-art results for RGB-only approaches
on the challenging YCB-V dataset. We illustrate the suitability of the
resulting model as pose sensor for a 6-DoF state estimation task. Code is
available at https://github.com/aau-cns/poet.
