  Designing robotic agents to perform open vocabulary tasks has been the
long-standing goal in robotics and AI. Recently, Large Language Models (LLMs)
have achieved impressive results in creating robotic agents for performing open
vocabulary tasks. However, planning for these tasks in the presence of
uncertainties is challenging as it requires \enquote{chain-of-thought}
reasoning, aggregating information from the environment, updating state
estimates, and generating actions based on the updated state estimates. In this
paper, we present an interactive planning technique for partially observable
tasks using LLMs. In the proposed method, an LLM is used to collect missing
information from the environment using a robot and infer the state of the
underlying problem from collected observations while guiding the robot to
perform the required actions. We also use a fine-tuned Llama 2 model via
self-instruct and compare its performance against a pre-trained LLM like GPT-4.
Results are demonstrated on several tasks in simulation as well as real-world
environments. A video describing our work along with some results could be
found here.
