  Existing text-to-image models still struggle to generate images of multiple
objects, especially in handling their spatial positions, relative sizes,
overlapping, and attribute bindings. To efficiently address these challenges,
we develop a training-free Multimodal-LLM agent (MuLan), as a human painter,
that can progressively generate multi-object with intricate planning and
feedback control. MuLan harnesses a large language model (LLM) to decompose a
prompt to a sequence of sub-tasks, each generating only one object by stable
diffusion, conditioned on previously generated objects. Unlike existing
LLM-grounded methods, MuLan only produces a high-level plan at the beginning
while the exact size and location of each object are determined upon each
sub-task by an LLM and attention guidance. Moreover, MuLan adopts a
vision-language model (VLM) to provide feedback to the image generated in each
sub-task and control the diffusion model to re-generate the image if it
violates the original prompt. Hence, each model in every step of MuLan only
needs to address an easy sub-task it is specialized for. The multi-step process
also allows human users to monitor the generation process and make preferred
changes at any intermediate step via text prompts, thereby improving the
human-AI collaboration experience. We collect 200 prompts containing
multi-objects with spatial relationships and attribute bindings from different
benchmarks to evaluate MuLan. The results demonstrate the superiority of MuLan
in generating multiple objects over baselines and its creativity when
collaborating with human users. The code is available at
https://github.com/measure-infinity/mulan-code.
