  We introduce SelfExplain, a novel self-explaining model that explains a text
classifier's predictions using phrase-based concepts. SelfExplain augments
existing neural classifiers by adding (1) a globally interpretable layer that
identifies the most influential concepts in the training set for a given sample
and (2) a locally interpretable layer that quantifies the contribution of each
local input concept by computing a relevance score relative to the predicted
label. Experiments across five text-classification datasets show that
SelfExplain facilitates interpretability without sacrificing performance. Most
importantly, explanations from SelfExplain show sufficiency for model
predictions and are perceived as adequate, trustworthy and understandable by
human judges compared to existing widely-used baselines.
