  Automated disease classification of radiology images has been emerging as a
promising technique to support clinical diagnosis and treatment planning.
Unlike generic image classification tasks, a real-world radiology image
classification task is significantly more challenging as it is far more
expensive to collect the training data where the labeled data is in nature
multi-label; and more seriously samples from easy classes often dominate;
training data is highly class-imbalanced problem exists in practice as well. To
overcome these challenges, in this paper, we propose a novel scheme of
Cross-Attention Networks (CAN) for automated thoracic disease classification
from chest x-ray images, which can effectively excavate more meaningful
representation from data to boost the performance through cross-attention by
only image-level annotations. We also design a new loss function that beyond
cross-entropy loss to help cross-attention process and is able to overcome the
imbalance between classes and easy-dominated samples within each class. The
proposed method achieves state-of-the-art results.
