  Embodied Instruction Following (EIF) is the task of executing natural
language instructions by navigating and interacting with objects in 3D
environments. One of the primary challenges in EIF is compositional task
planning, which is often addressed with supervised or in-context learning with
labeled data. To this end, we introduce the Socratic Planner, the first
zero-shot planning method that infers without the need for any training data.
Socratic Planner first decomposes the instructions into substructural
information of the task through self-questioning and answering, translating it
into a high-level plan, i.e., a sequence of subgoals. Subgoals are executed
sequentially, with our visually grounded re-planning mechanism adjusting plans
dynamically through a dense visual feedback. We also introduce an evaluation
metric of high-level plans, RelaxedHLP, for a more comprehensive evaluation.
Experiments demonstrate the effectiveness of the Socratic Planner, achieving
competitive performance on both zero-shot and few-shot task planning in the
ALFRED benchmark, particularly excelling in tasks requiring higher-dimensional
inference. Additionally, a precise adjustments in the plan were achieved by
incorporating environmental visual information.
