  Non-rigid inter-modality registration can facilitate accurate information
fusion from different modalities, but it is challenging due to the very
different image appearances across modalities. In this paper, we propose to
train a non-rigid inter-modality image registration network, which can directly
predict the transformation field from the input multimodal images, such as CT
and MR images. In particular, the training of our inter-modality registration
network is supervised by intra-modality similarity metric based on the
available paired data, which is derived from a pre-aligned CT and MR dataset.
Specifically, in the training stage, to register the input CT and MR images,
their similarity is evaluated on the warped MR image and the MR image that is
paired with the input CT. So that, the intra-modality similarity metric can be
directly applied to measure whether the input CT and MR images are well
registered. Moreover, we use the idea of dual-modality fashion, in which we
measure the similarity on both CT modality and MR modality. In this way, the
complementary anatomies in both modalities can be jointly considered to more
accurately train the inter-modality registration network. In the testing stage,
the trained inter-modality registration network can be directly applied to
register the new multimodal images without any paired data. Experimental
results have shown that, the proposed method can achieve promising accuracy and
efficiency for the challenging non-rigid inter-modality registration task and
also outperforms the state-of-the-art approaches.
