  A desirable property of learning systems is to be both effective and
interpretable. Towards this goal, recent models have been proposed that first
generate an extractive explanation from the input text and then generate a
prediction on just the explanation called explain-then-predict models. These
models primarily consider the task input as a supervision signal in learning an
extractive explanation and do not effectively integrate rationales data as an
additional inductive bias to improve task performance. We propose a novel yet
simple approach ExPred, that uses multi-task learning in the explanation
generation phase effectively trading-off explanation and prediction losses. And
then we use another prediction network on just the extracted explanations for
optimizing the task performance. We conduct an extensive evaluation of our
approach on three diverse language datasets -- fact verification, sentiment
classification, and QA -- and find that we substantially outperform existing
approaches.
