  Attention mechanisms have played a crucial role in the development of complex
architectures such as Transformers in natural language processing. However,
Transformers remain hard to interpret and are considered as black-boxes. This
paper aims to assess how attention coefficients from Transformers can help in
providing interpretability. A new attention-based interpretability method
called CLaSsification-Attention (CLS-A) is proposed. CLS-A computes an
interpretability score for each word based on the attention coefficient
distribution related to the part specific to the classification task within the
Transformer architecture. A human-grounded experiment is conducted to evaluate
and compare CLS-A to other interpretability methods. The experimental protocol
relies on the capacity of an interpretability method to provide explanation in
line with human reasoning. Experiment design includes measuring reaction times
and correct response rates by human subjects. CLS-A performs comparably to
usual interpretability methods regarding average participant reaction time and
accuracy. The lower computational cost of CLS-A compared to other
interpretability methods and its availability by design within the classifier
make it particularly interesting. Data analysis also highlights the link
between the probability score of a classifier prediction and adequate
explanations. Finally, our work confirms the relevancy of the use of CLS-A and
shows to which extent self-attention contains rich information to explain
Transformer classifiers.
