  Acquiring and annotating sufficient labeled data is crucial in developing
accurate and robust learning-based models, but obtaining such data can be
challenging in many medical image segmentation tasks. One promising solution is
to synthesize realistic data with ground-truth mask annotations. However, no
prior studies have explored generating complete 3D volumetric images with
masks. In this paper, we present MedGen3D, a deep generative framework that can
generate paired 3D medical images and masks. First, we represent the 3D medical
data as 2D sequences and propose the Multi-Condition Diffusion Probabilistic
Model (MC-DPM) to generate multi-label mask sequences adhering to anatomical
geometry. Then, we use an image sequence generator and semantic diffusion
refiner conditioned on the generated mask sequences to produce realistic 3D
medical images that align with the generated masks. Our proposed framework
guarantees accurate alignment between synthetic images and segmentation maps.
Experiments on 3D thoracic CT and brain MRI datasets show that our synthetic
data is both diverse and faithful to the original data, and demonstrate the
benefits for downstream segmentation tasks. We anticipate that MedGen3D's
ability to synthesize paired 3D medical images and masks will prove valuable in
training deep learning models for medical imaging tasks.
