  In order to develop robots that can effectively serve as versatile and
capable home assistants, it is crucial for them to reliably perceive and
interact with a wide variety of objects across diverse environments. To this
end, we proposed Open Vocabulary Mobile Manipulation as a key benchmark task
for robotics: finding any object in a novel environment and placing it on any
receptacle surface within that environment. We organized a NeurIPS 2023
competition featuring both simulation and real-world components to evaluate
solutions to this task. Our baselines on the most challenging version of this
task, using real perception in simulation, achieved only an 0.8% success rate;
by the end of the competition, the best participants achieved an 10.8\% success
rate, a 13x improvement. We observed that the most successful teams employed a
variety of methods, yet two common threads emerged among the best solutions:
enhancing error detection and recovery, and improving the integration of
perception with decision-making processes. In this paper, we detail the results
and methodologies used, both in simulation and real-world settings. We discuss
the lessons learned and their implications for future research. Additionally,
we compare performance in real and simulated environments, emphasizing the
necessity for robust generalization to novel settings.
