  In this paper, we build upon the weakly-supervised generation mechanism of
intermediate attention maps in any convolutional neural networks and disclose
the effectiveness of attention modules more straightforwardly to fully exploit
their potential. Given an existing neural network equipped with arbitrary
attention modules, we introduce a meta critic network to evaluate the quality
of attention maps in the main network. Due to the discreteness of our designed
reward, the proposed learning method is arranged in a reinforcement learning
setting, where the attention actors and recurrent critics are alternately
optimized to provide instant critique and revision for the temporary attention
representation, hence coined as Deep REinforced Attention Learning (DREAL). It
could be applied universally to network architectures with different types of
attention modules and promotes their expressive ability by maximizing the
relative gain of the final recognition performance arising from each individual
attention module, as demonstrated by extensive experiments on both category and
instance recognition benchmarks.
