  Visual explanation enables human to understand the decision making of Deep
Convolutional Neural Network (CNN), but it is insufficient to contribute the
performance improvement. In this paper, we focus on the attention map for
visual explanation, which represents high response value as the important
region in image recognition. This region significantly improves the performance
of CNN by introducing an attention mechanism that focuses on a specific region
in an image. In this work, we propose Attention Branch Network (ABN), which
extends the top-down visual explanation model by introducing a branch structure
with an attention mechanism. ABN can be applicable to several image recognition
tasks by introducing a branch for attention mechanism and is trainable for the
visual explanation and image recognition in end-to-end manner. We evaluate ABN
on several image recognition tasks such as image classification, fine-grained
recognition, and multiple facial attributes recognition. Experimental results
show that ABN can outperform the accuracy of baseline models on these image
recognition tasks while generating an attention map for visual explanation. Our
code is available at
https://github.com/machine-perception-robotics-group/attention_branch_network.
