  Magnetic resonance imaging (MRI) is the most sensitive technique for breast
cancer detection among current clinical imaging modalities. Contrast-enhanced
MRI (CE-MRI) provides superior differentiation between tumors and invaded
healthy tissue, and has become an indispensable technique in the detection and
evaluation of cancer. However, the use of gadolinium-based contrast agents
(GBCA) to obtain CE-MRI may be associated with nephrogenic systemic fibrosis
and may lead to bioaccumulation in the brain, posing a potential risk to human
health. Moreover, and likely more important, the use of gadolinium-based
contrast agents requires the cannulation of a vein, and the injection of the
contrast media which is cumbersome and places a burden on the patient. To
reduce the use of contrast agents, diffusion-weighted imaging (DWI) is emerging
as a key imaging technique, although currently usually complementing breast
CE-MRI. In this study, we develop a multi-sequence fusion network to synthesize
CE-MRI based on T1-weighted MRI and DWIs. DWIs with different b-values are
fused to efficiently utilize the difference features of DWIs. Rather than
proposing a pure data-driven approach, we invent a multi-sequence attention
module to obtain refined feature maps, and leverage hierarchical representation
information fused at different scales while utilizing the contributions from
different sequences from a model-driven approach by introducing the weighted
difference module. The results show that the multi-b-value DWI-based fusion
model can potentially be used to synthesize CE-MRI, thus theoretically reducing
or avoiding the use of GBCA, thereby minimizing the burden to patients. Our
code is available at
\url{https://github.com/Netherlands-Cancer-Institute/CE-MRI}.
