  When we deploy machine learning models in high-stakes medical settings, we
must ensure these models make accurate predictions that are consistent with
known medical science. Inherently interpretable networks address this need by
explaining the rationale behind each decision while maintaining equal or higher
accuracy compared to black-box models. In this work, we present a novel
interpretable neural network algorithm that uses case-based reasoning for
mammography. Designed to aid a radiologist in their decisions, our network
presents both a prediction of malignancy and an explanation of that prediction
using known medical features. In order to yield helpful explanations, the
network is designed to mimic the reasoning processes of a radiologist: our
network first detects the clinically relevant semantic features of each image
by comparing each new image with a learned set of prototypical image parts from
the training images, then uses those clinical features to predict malignancy.
Compared to other methods, our model detects clinical features (mass margins)
with equal or higher accuracy, provides a more detailed explanation of its
prediction, and is better able to differentiate the classification-relevant
parts of the image.
