  Recent advancements in text-to-image generation using diffusion models have
significantly improved the quality of generated images and expanded the ability
to depict a wide range of objects. However, ensuring that these models adhere
closely to the text prompts remains a considerable challenge. This issue is
particularly pronounced when trying to generate photorealistic images of
humans. Without significant prompt engineering efforts models often produce
unrealistic images and typically fail to incorporate the full extent of the
prompt information. This limitation can be largely attributed to the nature of
captions accompanying the images used in training large scale diffusion models,
which typically prioritize contextual information over details related to the
person's appearance. In this paper we address this issue by introducing a
training-free pipeline designed to generate accurate appearance descriptions
from images of people. We apply this method to create approximately 250,000
captions for publicly available face datasets. We then use these synthetic
captions to fine-tune a text-to-image diffusion model. Our results demonstrate
that this approach significantly improves the model's ability to generate
high-quality, realistic human faces and enhances adherence to the given
prompts, compared to the baseline model. We share our synthetic captions,
pretrained checkpoints and training code.
