  Deep learning methods have impacted almost every research field,
demonstrating notable successes in medical imaging tasks such as denoising and
super-resolution. However, the prerequisite for deep learning is data at scale,
but data sharing is expensive yet at risk of privacy leakage. As cutting-edge
AI generative models, diffusion models have now become dominant because of
their rigorous foundation and unprecedented outcomes. Here we propose a latent
diffusion approach for data synthesis without compromising patient privacy. In
our exemplary case studies, we develop a latent diffusion model to generate
medical CT, MRI and PET images using publicly available datasets. We
demonstrate that state-of-the-art deep learning-based
denoising/super-resolution networks can be trained on our synthetic data to
achieve image quality equivalent to what the same network can achieve after
being trained on the original data (the p values well exceeding the threshold
of 0.05). In our advanced diffusion model, we specifically embed a safeguard
mechanism to protect patient privacy effectively and efficiently. Consequently,
every synthetic image is guaranteed to be different by a pre-specified
threshold from the closest counterpart in the original patient dataset. Our
approach allows privacy-proof public sharing of diverse big datasets for
development of deep models, potentially enabling federated learning at the
level of input data instead of local network weights.
