  Transformers have supplanted Recurrent Neural Networks as the dominant
architecture for both natural language processing tasks and, despite criticisms
of cognitive implausibility, for modelling the effect of predictability on
online human language comprehension. However, two recently developed recurrent
neural network architectures, RWKV and Mamba, appear to perform natural
language tasks comparably to or better than transformers of equivalent scale.
In this paper, we show that contemporary recurrent models are now also able to
match - and in some cases, exceed - performance of comparably sized
transformers at modeling online human language comprehension. This suggests
that transformer language models are not uniquely suited to this task, and
opens up new directions for debates about the extent to which architectural
features of language models make them better or worse models of human language
comprehension.
