  Text-to-image generation has important implications for generation of diverse
and controllable images. Several attempts have been made to adapt Stable
Diffusion (SD) to the medical domain. However, the large distribution
difference between medical reports and natural texts, as well as high
computational complexity in common stable diffusion limit the authenticity and
feasibility of the generated medical images. To solve above problems, we
propose a novel light-weight transformer-based diffusion model learning
framework, Chest-Diffusion, for report-to-CXR generation. Chest-Diffusion
employs a domain-specific text encoder to obtain accurate and expressive text
features to guide image generation, improving the authenticity of the generated
images. Meanwhile, we introduce a light-weight transformer architecture as the
denoising model, reducing the computational complexity of the diffusion model.
Experiments demonstrate that our Chest-Diffusion achieves the lowest FID score
24.456, under the computation budget of 118.918 GFLOPs, which is nearly
one-third of the computational complexity of SD.
