  While there is increasing concern about the interpretability of neural
models, the evaluation of interpretability remains an open problem, due to the
lack of proper evaluation datasets and metrics. In this paper, we present a
novel benchmark to evaluate the interpretability of both neural models and
saliency methods. This benchmark covers three representative NLP tasks:
sentiment analysis, textual similarity and reading comprehension, each provided
with both English and Chinese annotated data. In order to precisely evaluate
the interpretability, we provide token-level rationales that are carefully
annotated to be sufficient, compact and comprehensive. We also design a new
metric, i.e., the consistency between the rationales before and after
perturbations, to uniformly evaluate the interpretability on different types of
tasks. Based on this benchmark, we conduct experiments on three typical models
with three saliency methods, and unveil their strengths and weakness in terms
of interpretability. We will release this benchmark
https://www.luge.ai/#/luge/task/taskDetail?taskId=15 and hope it can facilitate
the research in building trustworthy systems.
