  Recent advancements in text-to-image generation have been propelled by the
development of diffusion models and multi-modality learning. However, since
text is typically represented sequentially in these models, it often falls
short in providing accurate contextualization and structural control. So the
generated images do not consistently align with human expectations, especially
in complex scenarios involving multiple objects and relationships. In this
paper, we introduce the Scene Graph Adapter(SG-Adapter), leveraging the
structured representation of scene graphs to rectify inaccuracies in the
original text embeddings. The SG-Adapter's explicit and non-fully connected
graph representation greatly improves the fully connected, transformer-based
text representations. This enhancement is particularly notable in maintaining
precise correspondence in scenarios involving multiple relationships. To
address the challenges posed by low-quality annotated datasets like Visual
Genome, we have manually curated a highly clean, multi-relational scene
graph-image paired dataset MultiRels. Furthermore, we design three metrics
derived from GPT-4V to effectively and thoroughly measure the correspondence
between images and scene graphs. Both qualitative and quantitative results
validate the efficacy of our approach in controlling the correspondence in
multiple relationships.
