  We present a cross-modality generation framework that learns to generate
translated modalities from given modalities in MR images without real
acquisition. Our proposed method performs NeuroImage-to-NeuroImage translation
(abbreviated as N2N) by means of a deep learning model that leverages
conditional generative adversarial networks (cGANs). Our framework jointly
exploits the low-level features (pixel-wise information) and high-level
representations (e.g. brain tumors, brain structure like gray matter, etc.)
between cross modalities which are important for resolving the challenging
complexity in brain structures. Our framework can serve as an auxiliary method
in clinical diagnosis and has great application potential. Based on our
proposed framework, we first propose a method for cross-modality registration
by fusing the deformation fields to adopt the cross-modality information from
translated modalities. Second, we propose an approach for MRI segmentation,
translated multichannel segmentation (TMS), where given modalities, along with
translated modalities, are segmented by fully convolutional networks (FCN) in a
multichannel manner. Both of these two methods successfully adopt the
cross-modality information to improve the performance without adding any extra
data. Experiments demonstrate that our proposed framework advances the
state-of-the-art on five brain MRI datasets. We also observe encouraging
results in cross-modality registration and segmentation on some widely adopted
brain datasets. Overall, our work can serve as an auxiliary method in clinical
diagnosis and be applied to various tasks in medical fields.
  Keywords: image-to-image, cross-modality, registration, segmentation, brain
MRI
