  Multi-contrast MRI protocols increase the level of morphological information
available for diagnosis. Yet, the number and quality of contrasts is limited in
practice by various factors including scan time and patient motion. Synthesis
of missing or corrupted contrasts can alleviate this limitation to improve
clinical utility. Common approaches for multi-contrast MRI involve either
one-to-one and many-to-one synthesis methods. One-to-one methods take as input
a single source contrast, and they learn a latent representation sensitive to
unique features of the source. Meanwhile, many-to-one methods receive multiple
distinct sources, and they learn a shared latent representation more sensitive
to common features across sources. For enhanced image synthesis, here we
propose a multi-stream approach that aggregates information across multiple
source images via a mixture of multiple one-to-one streams and a joint
many-to-one stream. The shared feature maps generated in the many-to-one stream
and the complementary feature maps generated in the one-to-one streams are
combined with a fusion block. The location of the fusion block is adaptively
modified to maximize task-specific performance. Qualitative and quantitative
assessments on T1-, T2-, PD-weighted and FLAIR images clearly demonstrate the
superior performance of the proposed method compared to previous
state-of-the-art one-to-one and many-to-one methods.
