  Deep learning models designed for visual classification tasks on natural
images have become prevalent in medical image analysis. However, medical images
differ from typical natural images in many ways, such as significantly higher
resolutions and smaller regions of interest. Moreover, both the global
structure and local details play important roles in medical image analysis
tasks. To address these unique properties of medical images, we propose a
neural network that is able to classify breast cancer lesions utilizing
information from both a global saliency map and multiple local patches. The
proposed model outperforms the ResNet-based baseline and achieves
radiologist-level performance in the interpretation of screening mammography.
Although our model is trained only with image-level labels, it is able to
generate pixel-level saliency maps that provide localization of possible
malignant findings.
