  Recent advancements in diffusion models have significantly impacted the
trajectory of generative machine learning research, with many adopting the
strategy of fine-tuning pre-trained models using domain-specific text-to-image
datasets. Notably, this method has been readily employed for medical
applications, such as X-ray image synthesis, leveraging the plethora of
associated radiology reports. Yet, a prevailing concern is the lack of
assurance on whether these models genuinely comprehend their generated content.
With the evolution of text-conditional image generation, these models have
grown potent enough to facilitate object localization scrutiny. Our research
underscores this advancement in the critical realm of medical imaging,
emphasizing the crucial role of interpretability. We further unravel a
consequential trade-off between image fidelity as gauged by conventional
metrics and model interpretability in generative diffusion models.
Specifically, the adoption of learnable text encoders when fine-tuning results
in diminished interpretability. Our in-depth exploration uncovers the
underlying factors responsible for this divergence. Consequently, we present a
set of design principles for the development of truly interpretable generative
models. Code is available at https://github.com/MischaD/chest-distillation.
