  Given image labels as the only supervisory signal, we focus on harvesting, or
mining, thoracic disease localizations from chest X-ray images. Harvesting such
localizations from existing datasets allows for the creation of improved data
sources for computer-aided diagnosis and retrospective analyses. We train a
convolutional neural network (CNN) for image classification and propose an
attention mining (AM) strategy to improve the model's sensitivity or saliency
to disease patterns. The intuition of AM is that once the most salient disease
area is blocked or hidden from the CNN model, it will pay attention to
alternative image regions, while still attempting to make correct predictions.
However, the model requires to be properly constrained during AM, otherwise, it
may overfit to uncorrelated image parts and forget the valuable knowledge that
it has learned from the original image classification task. To alleviate such
side effects, we then design a knowledge preservation (KP) loss, which
minimizes the discrepancy between responses for X-ray images from the original
and the updated networks. Furthermore, we modify the CNN model to include
multi-scale aggregation (MSA), improving its localization ability on
small-scale disease findings, e.g., lung nodules. We experimentally validate
our method on the publicly-available ChestX-ray14 dataset, outperforming a
class activation map (CAM)-based approach, and demonstrating the value of our
novel framework for mining disease locations.
