  Diffusion model is a promising approach to image generation and has been
employed for Pose-Guided Person Image Synthesis (PGPIS) with competitive
performance. While existing methods simply align the person appearance to the
target pose, they are prone to overfitting due to the lack of a high-level
semantic understanding on the source person image. In this paper, we propose a
novel Coarse-to-Fine Latent Diffusion (CFLD) method for PGPIS. In the absence
of image-caption pairs and textual prompts, we develop a novel training
paradigm purely based on images to control the generation process of a
pre-trained text-to-image diffusion model. A perception-refined decoder is
designed to progressively refine a set of learnable queries and extract
semantic understanding of person images as a coarse-grained prompt. This allows
for the decoupling of fine-grained appearance and pose information controls at
different stages, and thus circumventing the potential overfitting problem. To
generate more realistic texture details, a hybrid-granularity attention module
is proposed to encode multi-scale fine-grained appearance features as bias
terms to augment the coarse-grained prompt. Both quantitative and qualitative
experimental results on the DeepFashion benchmark demonstrate the superiority
of our method over the state of the arts for PGPIS. Code is available at
https://github.com/YanzuoLu/CFLD.
