  Recent studies suggest that combined analysis of Magnetic resonance
imaging~(MRI) that measures brain atrophy and positron emission
tomography~(PET) that quantifies hypo-metabolism provides improved accuracy in
diagnosing Alzheimer's disease. However, such techniques are limited by the
availability of corresponding scans of each modality. Current work focuses on a
cross-modal approach to estimate FDG-PET scans for the given MR scans using a
3D U-Net architecture. The use of the complete MR image instead of a local
patch based approach helps in capturing non-local and non-linear correlations
between MRI and PET modalities. The quality of the estimated PET scans is
measured using quantitative metrics such as MAE, PSNR and SSIM. The efficacy of
the proposed method is evaluated in the context of Alzheimer's disease
classification. The accuracy using only MRI is 70.18% while joint
classification using synthesized PET and MRI is 74.43% with a p-value of
$0.06$. The significant improvement in diagnosis demonstrates the utility of
the synthesized PET scans for multi-modal analysis.
