  Recent works have shown that Large Language Models (LLMs) can facilitate the
grounding of instructions for robotic task planning. Despite this progress,
most existing works have primarily focused on utilizing raw images to aid LLMs
in understanding environmental information. However, this approach not only
limits the scope of observation but also typically necessitates extensive
multimodal data collection and large-scale models. In this paper, we propose a
novel approach called Graph-based Robotic Instruction Decomposer (GRID), which
leverages scene graphs instead of images to perceive global scene information
and iteratively plan subtasks for a given instruction. Our method encodes
object attributes and relationships in graphs through an LLM and Graph
Attention Networks, integrating instruction features to predict subtasks
consisting of pre-defined robot actions and target objects in the scene graph.
This strategy enables robots to acquire semantic knowledge widely observed in
the environment from the scene graph. To train and evaluate GRID, we establish
a dataset construction pipeline to generate synthetic datasets for graph-based
robotic task planning. Experiments have shown that our method outperforms GPT-4
by over 25.4% in subtask accuracy and 43.6% in task accuracy. Moreover, our
method achieves a real-time speed of 0.11s per inference. Experiments conducted
on datasets of unseen scenes and scenes with varying numbers of objects
demonstrate that the task accuracy of GRID declined by at most 3.8%, showcasing
its robust cross-scene generalization ability. We validate our method in both
physical simulation and the real world. More details can be found on the
project page https://jackyzengl.github.io/GRID.github.io/.
