  We propose DeepMiner, a framework to discover interpretable representations
in deep neural networks and to build explanations for medical predictions. By
probing convolutional neural networks (CNNs) trained to classify cancer in
mammograms, we show that many individual units in the final convolutional layer
of a CNN respond strongly to diseased tissue concepts specified by the BI-RADS
lexicon. After expert annotation of the interpretable units, our proposed
method is able to generate explanations for CNN mammogram classification that
are consistent with ground truth radiology reports on the Digital Database for
Screening Mammography. We show that DeepMiner not only enables better
understanding of the nuances of CNN classification decisions but also possibly
discovers new visual knowledge relevant to medical diagnosis.
