  It's been said that "Language Models are Unsupervised Multitask Learners."
Indeed, self-supervised language models trained on "positive" examples of
English text generalize in desirable ways to many natural language tasks. But
if such models can stray so far from an initial self-supervision objective, a
wayward model might generalize in undesirable ways too, say to nonsensical
"negative" examples of unnatural language. A key question in this work is: do
language models trained on (positive) training data also generalize to
(negative) test data? We use this question as a contrivance to assess the
extent to which language models learn undesirable properties of text, such as
n-grams, that might interfere with the learning of more desirable properties of
text, such as syntax. We find that within a model family, as the number of
parameters, training epochs, and data set size increase, so does a model's
ability to generalize to negative n-gram data, indicating standard
self-supervision generalizes too far. We propose a form of inductive bias that
attenuates such undesirable signals with negative data distributions
automatically learned from positive data. We apply the method to remove n-gram
signals from LSTMs and find that doing so causes them to favor syntactic
signals, as demonstrated by large error reductions (up to 46% on the hardest
cases) on a syntactic subject-verb agreement task.
