  Vision-language models (VLMs) have been applied to robot task planning
problems, where the robot receives a task in natural language and generates
plans based on visual inputs. While current VLMs have demonstrated strong
vision-language understanding capabilities, their performance is still far from
being satisfactory in planning tasks. At the same time, although classical task
planners, such as PDDL-based, are strong in planning for long-horizon tasks,
they do not work well in open worlds where unforeseen situations are common. In
this paper, we propose a novel task planning and execution framework, called
DKPROMPT, which automates VLM prompting using domain knowledge in PDDL for
classical planning in open worlds. Results from quantitative experiments show
that DKPROMPT outperforms classical planning, pure VLM-based and a few other
competitive baselines in task completion rate.
