  Generally, the small size of public medical imaging datasets coupled with
stringent privacy concerns, hampers the advancement of data-hungry deep
learning models in medical imaging. This study addresses these challenges for
3D cardiac MRI images in the short-axis view. We propose Latent Diffusion
Models that generate synthetic images conditioned on medical attributes, while
ensuring patient privacy through differentially private model training. To our
knowledge, this is the first work to apply and quantify differential privacy in
3D medical image generation. We pre-train our models on public data and
finetune them with differential privacy on the UK Biobank dataset. Our
experiments reveal that pre-training significantly improves model performance,
achieving a Fr\'echet Inception Distance (FID) of 26.77 at $\epsilon=10$,
compared to 92.52 for models without pre-training. Additionally, we explore the
trade-off between privacy constraints and image quality, investigating how
tighter privacy budgets affect output controllability and may lead to degraded
performance. Our results demonstrate that proper consideration during training
with differential privacy can substantially improve the quality of synthetic
cardiac MRI images, but there are still notable challenges in achieving
consistent medical realism.
