  Lack of ground-truth MR images impedes the common supervised training of
neural networks for image reconstruction. To cope with this challenge, this
paper leverages unpaired adversarial training for reconstruction networks,
where the inputs are undersampled k-space and naively reconstructed images from
one dataset, and the labels are high-quality images from another dataset. The
reconstruction networks consist of a generator which suppresses the input image
artifacts, and a discriminator using a pool of (unpaired) labels to adjust the
reconstruction quality. The generator is an unrolled neural network -- a
cascade of convolutional and data consistency layers. The discriminator is also
a multilayer CNN that plays the role of a critic scoring the quality of
reconstructed images based on the Wasserstein distance. Our experiments with
knee MRI datasets demonstrate that the proposed unpaired training enables
diagnostic-quality reconstruction when high-quality image labels are not
available for the input types of interest, or when the amount of labels is
small. In addition, our adversarial training scheme can achieve better image
quality (as rated by expert radiologists) compared with the paired training
schemes with pixel-wise loss.
