  Embodied AI focuses on the study and development of intelligent systems that
possess a physical or virtual embodiment (i.e. robots) and are able to
dynamically interact with their environment. Memory and control are the two
essential parts of an embodied system and usually require separate frameworks
to model each of them. In this paper, we propose a novel and generalizable
framework called LLM-Brain: using Large-scale Language Model as a robotic brain
to unify egocentric memory and control. The LLM-Brain framework integrates
multiple multimodal language models for robotic tasks, utilizing a zero-shot
learning approach. All components within LLM-Brain communicate using natural
language in closed-loop multi-round dialogues that encompass perception,
planning, control, and memory. The core of the system is an embodied LLM to
maintain egocentric memory and control the robot. We demonstrate LLM-Brain by
examining two downstream tasks: active exploration and embodied question
answering. The active exploration tasks require the robot to extensively
explore an unknown environment within a limited number of actions. Meanwhile,
the embodied question answering tasks necessitate that the robot answers
questions based on observations acquired during prior explorations.
