  We study the influence of context on sentence acceptability. First we compare
the acceptability ratings of sentences judged in isolation, with a relevant
context, and with an irrelevant context. Our results show that context induces
a cognitive load for humans, which compresses the distribution of ratings.
Moreover, in relevant contexts we observe a discourse coherence effect which
uniformly raises acceptability. Next, we test unidirectional and bidirectional
language models in their ability to predict acceptability ratings. The
bidirectional models show very promising results, with the best model achieving
a new state-of-the-art for unsupervised acceptability prediction. The two sets
of experiments provide insights into the cognitive aspects of sentence
processing and central issues in the computational modelling of text and
discourse.
