  Current Explainable AI (ExAI) methods, especially in the NLP field, are
conducted on various datasets by employing different metrics to evaluate
several aspects. The lack of a common evaluation framework is hindering the
progress tracking of such methods and their wider adoption. In this work,
inspired by offline information retrieval, we propose different metrics and
techniques to evaluate the explainability of SA models from two angles. First,
we evaluate the strength of the extracted "rationales" in faithfully explaining
the predicted outcome. Second, we measure the agreement between ExAI methods
and human judgment on a homegrown dataset1 to reflect on the rationales
plausibility. Our conducted experiments comprise four dimensions: (1) the
underlying architectures of SA models, (2) the approach followed by the ExAI
method, (3) the reasoning difficulty, and (4) the homogeneity of the
ground-truth rationales. We empirically demonstrate that anchors explanations
are more aligned with the human judgment and can be more confident in
extracting supporting rationales. As can be foreseen, the reasoning complexity
of sentiment is shown to thwart ExAI methods from extracting supporting
evidence. Moreover, a remarkable discrepancy is discerned between the results
of different explainability methods on the various architectures suggesting the
need for consolidation to observe enhanced performance. Predominantly,
transformers are shown to exhibit better explainability than convolutional and
recurrent architectures. Our work paves the way towards designing more
interpretable NLP models and enabling a common evaluation ground for their
relative strengths and robustness.
