  Despite being designed for performance rather than cognitive plausibility,
transformer language models have been found to be better at predicting metrics
used to assess human language comprehension than language models with other
architectures, such as recurrent neural networks. Based on how well they
predict the N400, a neural signal associated with processing difficulty, we
propose and provide evidence for one possible explanation - their predictions
are affected by the preceding context in a way analogous to the effect of
semantic facilitation in humans.
