  Language models (LMs) have been used in cognitive modeling as well as
engineering studies -- they compute information-theoretic complexity metrics
that simulate humans' cognitive load during reading. This study highlights a
limitation of modern neural LMs as the model of choice for this purpose: there
is a discrepancy between their context access capacities and that of humans.
Our results showed that constraining the LMs' context access improved their
simulation of human reading behavior. We also showed that LM-human gaps in
context access were associated with specific syntactic constructions;
incorporating syntactic biases into LMs' context access might enhance their
cognitive plausibility.
