  Fusing structural-functional images of the brain has shown great potential to
analyze the deterioration of Alzheimer's disease (AD). However, it is a big
challenge to effectively fuse the correlated and complementary information from
multimodal neuroimages. In this paper, a novel model termed cross-modal
transformer generative adversarial network (CT-GAN) is proposed to effectively
fuse the functional and structural information contained in functional magnetic
resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can
learn topological features and generate multimodal connectivity from multimodal
imaging data in an efficient end-to-end manner. Moreover, the swapping
bi-attention mechanism is designed to gradually align common features and
effectively enhance the complementary features between modalities. By analyzing
the generated connectivity features, the proposed model can identify AD-related
brain connections. Evaluations on the public ADNI dataset show that the
proposed CT-GAN can dramatically improve prediction performance and detect
AD-related brain regions effectively. The proposed model also provides new
insights for detecting AD-related abnormal neural circuits.
