  Recent developments in gradient-based attention modeling have seen attention
maps emerge as a powerful tool for interpreting convolutional neural networks.
Despite good localization for an individual class of interest, these techniques
produce attention maps with substantially overlapping responses among different
classes, leading to the problem of visual confusion and the need for
discriminative attention. In this paper, we address this problem by means of a
new framework that makes class-discriminative attention a principled part of
the learning process. Our key innovations include new learning objectives for
attention separability and cross-layer consistency, which result in improved
attention discriminability and reduced visual confusion. Extensive experiments
on image classification benchmarks show the effectiveness of our approach in
terms of improved classification accuracy, including CIFAR-100 (+3.33%),
Caltech-256 (+1.64%), ILSVRC2012 (+0.92%), CUB-200-2011 (+4.8%) and PASCAL
VOC2012 (+5.73%).
