  Deep Convolutional Neural Networks (CNNs) are becoming prominent models for
semi-automated diagnosis of Alzheimer's Disease (AD) using brain Magnetic
Resonance Imaging (MRI). Although being highly accurate, deep CNN models lack
transparency and interpretability, precluding adequate clinical reasoning and
not complying with most current regulatory demands. One popular choice for
explaining deep image models is occluding regions of the image to isolate their
influence on the prediction. However, existing methods for occluding patches of
brain scans generate images outside the distribution to which the model was
trained for, thus leading to unreliable explanations. In this paper, we propose
an alternative explanation method that is specifically designed for the brain
scan task. Our method, which we refer to as Swap Test, produces heatmaps that
depict the areas of the brain that are most indicative of AD, providing
interpretability for the model's decisions in a format understandable to
clinicians. Experimental results using an axiomatic evaluation show that the
proposed method is more suitable for explaining the diagnosis of AD using MRI
while the opposite trend was observed when using a typical occlusion test.
Therefore, we believe our method may address the inherent black-box nature of
deep neural networks that are capable of diagnosing AD.
