  In recent years, the number of papers on Alzheimer's disease classification
has increased dramatically, generating interesting methodological ideas on the
use machine learning and feature extraction methods. However, practical impact
is much more limited and, eventually, one could not tell which of these
approaches are the most efficient. While over 90\% of these works make use of
ADNI an objective comparison between approaches is impossible due to variations
in the subjects included, image pre-processing, performance metrics and
cross-validation procedures. In this paper, we propose a framework for
reproducible classification experiments using multimodal MRI and PET data from
ADNI. The core components are: 1) code to automatically convert the full ADNI
database into BIDS format; 2) a modular architecture based on Nipype in order
to easily plug-in different classification and feature extraction tools; 3)
feature extraction pipelines for MRI and PET data; 4) baseline classification
approaches for unimodal and multimodal features. This provides a flexible
framework for benchmarking different feature extraction and classification
tools in a reproducible manner. We demonstrate its use on all (1519) baseline
T1 MR images and all (1102) baseline FDG PET images from ADNI 1, GO and 2 with
SPM-based feature extraction pipelines and three different classification
techniques (linear SVM, anatomically regularized SVM and multiple kernel
learning SVM). The highest accuracies achieved were: 91% for AD vs CN, 83% for
MCIc vs CN, 75% for MCIc vs MCInc, 94% for AD-A$\beta$+ vs CN-A$\beta$- and 72%
for MCIc-A$\beta$+ vs MCInc-A$\beta$+. The code is publicly available at
https://gitlab.icm-institute.org/aramislab/AD-ML (depends on the Clinica
software platform, publicly available at http://www.clinica.run).
