  Locating objects referred to in natural language poses a significant
challenge for autonomous agents. Existing CLIP-based open-vocabulary methods
successfully perform 3D object retrieval with simple (bare) queries but cannot
cope with ambiguous descriptions that demand an understanding of object
relations. To tackle this problem, we propose a modular approach called BBQ
(Beyond Bare Queries), which constructs 3D scene spatial graph representation
with metric edges and utilizes a large language model as a human-to-agent
interface through our deductive scene reasoning algorithm. BBQ employs robust
DINO-powered associations to form 3D objects, an advanced raycasting algorithm
to project them to 2D, and a vision-language model to describe them as graph
nodes. On Replica and ScanNet datasets, we show that the designed method
accurately constructs 3D object-centric maps. We have demonstrated that their
quality takes a leading place for open-vocabulary 3D semantic segmentation
against other zero-shot methods. Also, we show that leveraging spatial
relations is especially effective for scenes containing multiple entities of
the same semantic class. On Sr3D and Nr3D benchmarks, our deductive approach
demonstrates a significant improvement, enabling retrieving objects by complex
queries compared to other state-of-the-art methods. Considering our design
solutions, we achieved a processing speed approximately x3 times faster than
the closest analog. This promising performance enables our approach for usage
in applied intelligent robotics projects. We make the code publicly available
at linukc.github.io/bbq/.
