  Localizing objects in 3D scenes based on natural language requires
understanding and reasoning about spatial relations. In particular, it is often
crucial to distinguish similar objects referred by the text, such as "the left
most chair" and "a chair next to the window". In this work we propose a
language-conditioned transformer model for grounding 3D objects and their
spatial relations. To this end, we design a spatial self-attention layer that
accounts for relative distances and orientations between objects in input 3D
point clouds. Training such a layer with visual and language inputs enables to
disambiguate spatial relations and to localize objects referred by the text. To
facilitate the cross-modal learning of relations, we further propose a
teacher-student approach where the teacher model is first trained using
ground-truth object labels, and then helps to train a student model using point
cloud inputs. We perform ablation studies showing advantages of our approach.
We also demonstrate our model to significantly outperform the state of the art
on the challenging Nr3D, Sr3D and ScanRefer 3D object grounding datasets.
