  The ability to accurately locate and navigate to a specific object is a
crucial capability for embodied agents that operate in the real world and
interact with objects to complete tasks. Such object navigation tasks usually
require large-scale training in visual environments with labeled objects, which
generalizes poorly to novel objects in unknown environments. In this work, we
present a novel zero-shot object navigation method, Exploration with Soft
Commonsense constraints (ESC), that transfers commonsense knowledge in
pre-trained models to open-world object navigation without any navigation
experience nor any other training on the visual environments. First, ESC
leverages a pre-trained vision and language model for open-world prompt-based
grounding and a pre-trained commonsense language model for room and object
reasoning. Then ESC converts commonsense knowledge into navigation actions by
modeling it as soft logic predicates for efficient exploration. Extensive
experiments on MP3D, HM3D, and RoboTHOR benchmarks show that our ESC method
improves significantly over baselines, and achieves new state-of-the-art
results for zero-shot object navigation (e.g., 288% relative Success Rate
improvement than CoW on MP3D).
