  We propose a novel Text-to-Image Generation Network, Adaptive Layout
Refinement Generative Adversarial Network (ALR-GAN), to adaptively refine the
layout of synthesized images without any auxiliary information. The ALR-GAN
includes an Adaptive Layout Refinement (ALR) module and a Layout Visual
Refinement (LVR) loss. The ALR module aligns the layout structure (which refers
to locations of objects and background) of a synthesized image with that of its
corresponding real image. In ALR module, we proposed an Adaptive Layout
Refinement (ALR) loss to balance the matching of hard and easy features, for
more efficient layout structure matching. Based on the refined layout
structure, the LVR loss further refines the visual representation within the
layout area. Experimental results on two widely-used datasets show that ALR-GAN
performs competitively at the Text-to-Image generation task.
