  While attention has been an increasingly popular component in deep neural
networks to both interpret and boost the performance of models, little work has
examined how attention progresses to accomplish a task and whether it is
reasonable. In this work, we propose an Attention with Reasoning capability
(AiR) framework that uses attention to understand and improve the process
leading to task outcomes. We first define an evaluation metric based on a
sequence of atomic reasoning operations, enabling a quantitative measurement of
attention that considers the reasoning process. We then collect human
eye-tracking and answer correctness data, and analyze various machine and human
attention mechanisms on their reasoning capability and how they impact task
performance. To improve the attention and reasoning ability of visual question
answering models, we propose to supervise the learning of attention
progressively along the reasoning process and to differentiate the correct and
incorrect attention patterns. We demonstrate the effectiveness of the proposed
framework in analyzing and modeling attention with better reasoning capability
and task performance. The code and data are available at
https://github.com/szzexpoi/AiR
