  Recent text-to-image (T2I) diffusion models show outstanding performance in
generating high-quality images conditioned on textual prompts. However, they
fail to semantically align the generated images with the prompts due to their
limited compositional capabilities, leading to attribute leakage, entity
leakage, and missing entities. In this paper, we propose a novel attention mask
control strategy based on predicted object boxes to address these issues. In
particular, we first train a BoxNet to predict a box for each entity that
possesses the attribute specified in the prompt. Then, depending on the
predicted boxes, a unique mask control is applied to the cross- and
self-attention maps. Our approach produces a more semantically accurate
synthesis by constraining the attention regions of each token in the prompt to
the image. In addition, the proposed method is straightforward and effective
and can be readily integrated into existing cross-attention-based T2I
generators. We compare our approach to competing methods and demonstrate that
it can faithfully convey the semantics of the original text to the generated
content and achieve high availability as a ready-to-use plugin. Please refer to
https://github.com/OPPOMente-Lab/attention-mask-control.
