  Multimodal Large Language Models (MLLMs) have shown impressive reasoning
abilities and general intelligence in various domains. It inspires researchers
to train end-to-end MLLMs or utilize large models to generate policies with
human-selected prompts for embodied agents. However, these methods exhibit
limited generalization capabilities on unseen tasks or scenarios, and overlook
the multimodal environment information which is critical for robots to make
decisions. In this paper, we introduce a novel Robotic Multimodal
Perception-Planning (RoboMP$^2$) framework for robotic manipulation which
consists of a Goal-Conditioned Multimodal Preceptor (GCMP) and a
Retrieval-Augmented Multimodal Planner (RAMP). Specially, GCMP captures
environment states by employing a tailored MLLMs for embodied agents with the
abilities of semantic reasoning and localization. RAMP utilizes coarse-to-fine
retrieval method to find the $k$ most-relevant policies as in-context
demonstrations to enhance the planner. Extensive experiments demonstrate the
superiority of RoboMP$^2$ on both VIMA benchmark and real-world tasks, with
around 10% improvement over the baselines.
