  Suppose we want to specify the inductive bias that married couples typically
go on honeymoons for the task of extracting pairs of spouses from text. In this
paper, we allow model developers to specify these types of inductive biases as
natural language explanations. We use BERT fine-tuned on MultiNLI to
``interpret'' these explanations with respect to the input sentence, producing
explanation-guided representations of the input. Across three relation
extraction tasks, our method, ExpBERT, matches a BERT baseline but with 3--20x
less labeled data and improves on the baseline by 3--10 F1 points with the same
amount of labeled data.
