  In recent years, model explanation methods have been designed to interpret
model decisions faithfully and intuitively so that users can easily understand
them. In this paper, we propose a framework, Faithful Attention Explainer
(FAE), capable of generating faithful textual explanations regarding the
attended-to features. Towards this goal, we deploy an attention module that
takes the visual feature maps from the classifier for sentence generation.
Furthermore, our method successfully learns the association between features
and words, which allows a novel attention enforcement module for attention
explanation. Our model achieves promising performance in caption quality
metrics and a faithful decision-relevance metric on two datasets (CUB and
ACT-X). In addition, we show that FAE can interpret gaze-based human attention,
as human gaze indicates the discriminative features that humans use for
decision-making, demonstrating the potential of deploying human gaze for
advanced human-AI interaction.
