  Multi-modality imaging improves disease diagnosis and reveals distinct
deviations in tissues with anatomical properties. The existence of completely
aligned and paired multi-modality neuroimaging data has proved its
effectiveness in brain research. However, collecting fully aligned and paired
data is expensive or even impractical, since it faces many difficulties,
including high cost, long acquisition time, image corruption, and privacy
issues. An alternative solution is to explore unsupervised or weakly supervised
learning methods to synthesize the absent neuroimaging data. In this paper, we
provide a comprehensive review of cross-modality synthesis for neuroimages,
from the perspectives of weakly supervised and unsupervised settings, loss
functions, evaluation metrics, imaging modalities, datasets, and downstream
applications based on synthesis. We begin by highlighting several opening
challenges for cross-modality neuroimage synthesis. Then, we discuss
representative architectures of cross-modality synthesis methods under
different supervisions. This is followed by a stepwise in-depth analysis to
evaluate how cross-modality neuroimage synthesis improves the performance of
its downstream tasks. Finally, we summarize the existing research findings and
point out future research directions. All resources are available at
https://github.com/M-3LAB/awesome-multimodal-brain-image-systhesis
