  Medical imaging plays a critical role in various clinical applications.
However, due to multiple considerations such as cost and risk, the acquisition
of certain image modalities could be limited. To address this issue, many
cross-modality medical image synthesis methods have been proposed. However, the
current methods cannot well model the hard-to-synthesis regions (e.g., tumor or
lesion regions). To address this issue, we propose a simple but effective
strategy, that is, we propose a dual-discriminator (dual-D) adversarial
learning system, in which, a global-D is used to make an overall evaluation for
the synthetic image, and a local-D is proposed to densely evaluate the local
regions of the synthetic image. More importantly, we build an adversarial
attention mechanism which targets at better modeling hard-to-synthesize regions
(e.g., tumor or lesion regions) based on the local-D. Experimental results show
the robustness and accuracy of our method in synthesizing fine-grained target
images from the corresponding source images. In particular, we evaluate our
method on two datasets, i.e., to address the tasks of generating T2 MRI from T1
MRI for the brain tumor images and generating MRI from CT. Our method
outperforms the state-of-the-art methods under comparison in all datasets and
tasks. And the proposed difficult-region-aware attention mechanism is also
proved to be able to help generate more realistic images, especially for the
hard-to-synthesize regions.
