  Most evaluations of attribution methods focus on the English language. In
this work, we present a multilingual approach for evaluating attribution
methods for the Natural Language Inference (NLI) task in terms of faithfulness
and plausibility. First, we introduce a novel cross-lingual strategy to measure
faithfulness based on word alignments, which eliminates the drawbacks of
erasure-based evaluations.We then perform a comprehensive evaluation of
attribution methods, considering different output mechanisms and aggregation
methods. Finally, we augment the XNLI dataset with highlight-based
explanations, providing a multilingual NLI dataset with highlights, to support
future exNLP studies. Our results show that attribution methods performing best
for plausibility and faithfulness are different.
