  The Eye Movements on Machine-Generated Texts Corpus (EMTeC) is a naturalistic
eye-movements-while-reading corpus of 107 native English speakers reading
machine-generated texts. The texts are generated by three large language models
using five different decoding strategies, and they fall into six different text
type categories. EMTeC entails the eye movement data at all stages of
pre-processing, i.e., the raw coordinate data sampled at 2000 Hz, the fixation
sequences, and the reading measures. It further provides both the original and
a corrected version of the fixation sequences, accounting for vertical
calibration drift. Moreover, the corpus includes the language models' internals
that underlie the generation of the stimulus texts: the transition scores, the
attention scores, and the hidden states. The stimuli are annotated for a range
of linguistic features both at text and at word level. We anticipate EMTeC to
be utilized for a variety of use cases such as, but not restricted to, the
investigation of reading behavior on machine-generated text and the impact of
different decoding strategies; reading behavior on different text types; the
development of new pre-processing, data filtering, and drift correction
algorithms; the cognitive interpretability and enhancement of language models;
and the assessment of the predictive power of surprisal and entropy for human
reading times. The data at all stages of pre-processing, the model internals,
and the code to reproduce the stimulus generation, data pre-processing and
analyses can be accessed via https://github.com/DiLi-Lab/EMTeC/.
