  Convolutional neural networks have enabled significant improvements in
medical image-based diagnosis. It is, however, increasingly clear that these
models are susceptible to performance degradation when facing spurious
correlations and dataset shift, leading, e.g., to underperformance on
underrepresented patient groups. In this paper, we compare two classification
schemes on the ADNI MRI dataset: a simple logistic regression model using
manually selected volumetric features, and a convolutional neural network
trained on 3D MRI data. We assess the robustness of the trained models in the
face of varying dataset splits, training set sex composition, and stage of
disease. In contrast to earlier work in other imaging modalities, we do not
observe a clear pattern of improved model performance for the majority group in
the training dataset. Instead, while logistic regression is fully robust to
dataset composition, we find that CNN performance is generally improved for
both male and female subjects when including more female subjects in the
training dataset. We hypothesize that this might be due to inherent differences
in the pathology of the two sexes. Moreover, in our analysis, the logistic
regression model outperforms the 3D CNN, emphasizing the utility of manual
feature specification based on prior knowledge, and the need for more robust
automatic feature selection.
