  Explaining the predictions of AI models is paramount in safety-critical
applications, such as in legal or medical domains. One form of explanation for
a prediction is an extractive rationale, i.e., a subset of features of an
instance that lead the model to give its prediction on that instance. For
example, the subphrase ``he stole the mobile phone'' can be an extractive
rationale for the prediction of ``Theft''. Previous works on generating
extractive rationales usually employ a two-phase model: a selector that selects
the most important features (i.e., the rationale) followed by a predictor that
makes the prediction based exclusively on the selected features. One
disadvantage of these works is that the main signal for learning to select
features comes from the comparison of the answers given by the predictor to the
ground-truth answers. In this work, we propose to squeeze more information from
the predictor via an information calibration method. More precisely, we train
two models jointly: one is a typical neural model that solves the task at hand
in an accurate but black-box manner, and the other is a selector-predictor
model that additionally produces a rationale for its prediction. The first
model is used as a guide for the second model. We use an adversarial technique
to calibrate the information extracted by the two models such that the
difference between them is an indicator of the missed or over-selected
features. In addition, for natural language tasks, we propose a
language-model-based regularizer to encourage the extraction of fluent
rationales. Experimental results on a sentiment analysis task, a hate speech
recognition task as well as on three tasks from the legal domain show the
effectiveness of our approach to rationale extraction.
