  Multi-label image classification is a fundamental but challenging task in
computer vision. Great progress has been achieved by exploiting semantic
relations between labels in recent years. However, conventional approaches are
unable to model the underlying spatial relations between labels in multi-label
images, because spatial annotations of the labels are generally not provided.
In this paper, we propose a unified deep neural network that exploits both
semantic and spatial relations between labels with only image-level
supervisions. Given a multi-label image, our proposed Spatial Regularization
Network (SRN) generates attention maps for all labels and captures the
underlying relations between them via learnable convolutions. By aggregating
the regularized classification results with original results by a ResNet-101
network, the classification performance can be consistently improved. The whole
deep neural network is trained end-to-end with only image-level annotations,
thus requires no additional efforts on image annotations. Extensive evaluations
on 3 public datasets with different types of labels show that our approach
significantly outperforms state-of-the-arts and has strong generalization
capability. Analysis of the learned SRN model demonstrates that it can
effectively capture both semantic and spatial relations of labels for improving
classification performance.
