  Deep neural network models have recently draw lots of attention, as it
consistently produce impressive results in many computer vision tasks such as
image classification, object detection, etc. However, interpreting such model
and show the reason why it performs quite well becomes a challenging question.
In this paper, we propose a novel method to interpret the neural network models
with attention mechanism. Inspired by the heatmap visualization, we analyze the
relation between classification accuracy with the attention based heatmap. An
improved attention based method is also included and illustrate that a better
classifier can be interpreted by the attention based heatmap.
