  While deep learning methods have shown great success in medical image
analysis, they require a number of medical images to train. Due to data privacy
concerns and unavailability of medical annotators, it is oftentimes very
difficult to obtain a lot of labeled medical images for model training. In this
paper, we study cross-modality data augmentation to mitigate the data
deficiency issue in the medical imaging domain. We propose a discriminative
unpaired image-to-image translation model which translates images in source
modality into images in target modality where the translation task is conducted
jointly with the downstream prediction task and the translation is guided by
the prediction. Experiments on two applications demonstrate the effectiveness
of our method.
