  Building systems that achieve a deeper understanding of language is one of
the central goals of natural language processing (NLP). Towards this goal,
recent works have begun to train language models on narrative datasets which
require extracting the most critical information by integrating across long
contexts. However, it is still an open question whether these models are
learning a deeper understanding of the text, or if the models are simply
learning a heuristic to complete the task. This work investigates this further
by turning to the one language processing system that truly understands complex
language: the human brain. We show that training language models for deeper
narrative understanding results in richer representations that have improved
alignment to human brain activity. We further find that the improvements in
brain alignment are larger for character names than for other discourse
features, which indicates that these models are learning important narrative
elements. Taken together, these results suggest that this type of training can
indeed lead to deeper language understanding. These findings have consequences
both for cognitive neuroscience by revealing some of the significant factors
behind brain-NLP alignment, and for NLP by highlighting that understanding of
long-range context can be improved beyond language modeling.
