  Text-to-point-cloud cross-modal localization is an emerging vision-language
task critical for future robot-human collaboration. It seeks to localize a
position from a city-scale point cloud scene based on a few natural language
instructions. In this paper, we address two key limitations of existing
approaches: 1) their reliance on ground-truth instances as input; and 2) their
neglect of the relative positions among potential instances. Our proposed model
follows a two-stage pipeline, including a coarse stage for text-cell retrieval
and a fine stage for position estimation. In both stages, we introduce an
instance query extractor, in which the cells are encoded by a 3D sparse
convolution U-Net to generate the multi-scale point cloud features, and a set
of queries iteratively attend to these features to represent instances. In the
coarse stage, a row-column relative position-aware self-attention (RowColRPA)
module is designed to capture the spatial relations among the instance queries.
In the fine stage, a multi-modal relative position-aware cross-attention (RPCA)
module is developed to fuse the text and point cloud features along with
spatial relations for improving fine position estimation. Experiment results on
the KITTI360Pose dataset demonstrate that our model achieves competitive
performance with the state-of-the-art models without taking ground-truth
instances as input.
