  With the explosive influence caused by the success of large language models
(LLM) like ChatGPT and GPT-4, there has been an extensive amount of recent work
showing that foundation models can be used to solve a large variety of tasks.
However, there is very limited work that shares insights on multi-agent
planning. Multi-agent planning is different from other domains by combining the
difficulty of multi-agent coordination and planning, and making it hard to
leverage external tools to facilitate the reasoning needed. In this paper, we
focus on the problem of multi-agent path finding (MAPF), which is also known as
multi-robot route planning, and study the performance of solving MAPF with
LLMs. We first show the motivating success on an empty room map without
obstacles, then the failure to plan on the harder room map and maze map of the
standard MAPF benchmark. We present our position on why directly solving MAPF
with LLMs has not been successful yet, and we use various experiments to
support our hypothesis. Based on our results, we discussed how researchers with
different backgrounds could help with this problem from different perspectives.
