  Generative Adversarial Networks (GAN) have many potential medical imaging
applications, including data augmentation, domain adaptation, and model
explanation. Due to the limited memory of Graphical Processing Units (GPUs),
most current 3D GAN models are trained on low-resolution medical images, these
models either cannot scale to high-resolution or are prone to patchy artifacts.
In this work, we propose a novel end-to-end GAN architecture that can generate
high-resolution 3D images. We achieve this goal by using different
configurations between training and inference. During training, we adopt a
hierarchical structure that simultaneously generates a low-resolution version
of the image and a randomly selected sub-volume of the high-resolution image.
The hierarchical design has two advantages: First, the memory demand for
training on high-resolution images is amortized among sub-volumes. Furthermore,
anchoring the high-resolution sub-volumes to a single low-resolution image
ensures anatomical consistency between sub-volumes. During inference, our model
can directly generate full high-resolution images. We also incorporate an
encoder with a similar hierarchical structure into the model to extract
features from the images. Experiments on 3D thorax CT and brain MRI demonstrate
that our approach outperforms state of the art in image generation. We also
demonstrate clinical applications of the proposed model in data augmentation
and clinical-relevant feature extraction.
