  Automated medical image classification is the key component in intelligent
diagnosis systems. However, most medical image datasets contain plenty of
samples of common diseases and just a handful of rare ones, leading to major
class imbalances. Currently, it is an open problem in intelligent diagnosis to
effectively learn from imbalanced training data. In this paper, we propose a
simple yet effective framework, named \textbf{C}lass \textbf{A}ttention to
\textbf{RE}gions of the lesion (CARE), to handle data imbalance issues by
embedding attention into the training process of \textbf{C}onvolutional
\textbf{N}eural \textbf{N}etworks (CNNs). The proposed attention module helps
CNNs attend to lesion regions of rare diseases, therefore helping CNNs to learn
their characteristics more effectively. In addition, this attention module
works only during the training phase and does not change the architecture of
the original network, so it can be directly combined with any existing CNN
architecture. The CARE framework needs bounding boxes to represent the lesion
regions of rare diseases. To alleviate the need for manual annotation, we
further developed variants of CARE by leveraging the traditional saliency
methods or a pretrained segmentation model for bounding box generation. Results
show that the CARE variants with automated bounding box generation are
comparable to the original CARE framework with \textit{manual} bounding box
annotations. A series of experiments on an imbalanced skin image dataset and a
pneumonia dataset indicates that our method can effectively help the network
focus on the lesion regions of rare diseases and remarkably improves the
classification performance of rare diseases.
