  Alzheimer's Disease (AD) is one of the most concerned neurodegenerative
diseases. In the last decade, studies on AD diagnosis attached great
significance to artificial intelligence (AI)-based diagnostic algorithms. Among
the diverse modality imaging data, T1-weighted MRI and 18F-FDGPET are widely
researched for this task. In this paper, we propose a novel convolutional
neural network (CNN) to fuse the multi-modality information including T1-MRI
and FDG-PDT images around the hippocampal area for the diagnosis of AD.
Different from the traditional machine learning algorithms, this method does
not require manually extracted features, and utilizes the stateof-art 3D
image-processing CNNs to learn features for the diagnosis and prognosis of AD.
To validate the performance of the proposed network, we trained the classifier
with paired T1-MRI and FDG-PET images using the ADNI datasets, including 731
Normal (NL) subjects, 647 AD subjects, 441 stable MCI (sMCI) subjects and 326
progressive MCI (pMCI) subjects. We obtained the maximal accuracies of 90.10%
for NL/AD task, 87.46% for NL/pMCI task, and 76.90% for sMCI/pMCI task. The
proposed framework yields comparative results against state-of-the-art
approaches. Moreover, the experimental results have demonstrated that (1)
segmentation is not a prerequisite by using CNN, (2) the hippocampal area
provides enough information to give a reference to AD diagnosis. Keywords:
Alzheimer's Disease, Multi-modality, Image Classification, CNN, Deep Learning,
Hippocampal
