  In this work, we consider the task of pairwise cross-modality image
registration, which may benefit from exploiting additional images available
only at training time from an additional modality that is different to those
being registered. As an example, we focus on aligning intra-subject
multiparametric Magnetic Resonance (mpMR) images, between T2-weighted (T2w)
scans and diffusion-weighted scans with high b-value (DWI$_{high-b}$). For the
application of localising tumours in mpMR images, diffusion scans with zero
b-value (DWI$_{b=0}$) are considered easier to register to T2w due to the
availability of corresponding features. We propose a learning from privileged
modality algorithm, using a training-only imaging modality DWI$_{b=0}$, to
support the challenging multi-modality registration problems. We present
experimental results based on 369 sets of 3D multiparametric MRI images from
356 prostate cancer patients and report, with statistical significance, a
lowered median target registration error of 4.34 mm, when registering the
holdout DWI$_{high-b}$ and T2w image pairs, compared with that of 7.96 mm
before registration. Results also show that the proposed learning-based
registration networks enabled efficient registration with comparable or better
accuracy, compared with a classical iterative algorithm and other tested
learning-based methods with/without the additional modality. These compared
algorithms also failed to produce any significantly improved alignment between
DWI$_{high-b}$ and T2w in this challenging application.
