  To build an interpretable neural text classifier, most of the prior work has
focused on designing inherently interpretable models or finding faithful
explanations. A new line of work on improving model interpretability has just
started, and many existing methods require either prior information or human
annotations as additional inputs in training. To address this limitation, we
propose the variational word mask (VMASK) method to automatically learn
task-specific important words and reduce irrelevant information on
classification, which ultimately improves the interpretability of model
predictions. The proposed method is evaluated with three neural text
classifiers (CNN, LSTM, and BERT) on seven benchmark text classification
datasets. Experiments show the effectiveness of VMASK in improving both model
prediction accuracy and interpretability.
