  From smoothly pursuing moving objects to rapidly shifting gazes during visual
search, humans employ a wide variety of eye movement strategies in different
contexts. While eye movements provide a rich window into mental processes,
building generative models of eye movements is notoriously difficult, and to
date the computational objectives guiding eye movements remain largely a
mystery. In this work, we tackled these problems in the context of a canonical
spatial planning task, maze-solving. We collected eye movement data from human
subjects and built deep generative models of eye movements using a novel
differentiable architecture for gaze fixations and gaze shifts. We found that
human eye movements are best predicted by a model that is optimized not to
perform the task as efficiently as possible but instead to run an internal
simulation of an object traversing the maze. This not only provides a
generative model of eye movements in this task but also suggests a
computational theory for how humans solve the task, namely that humans use
mental simulation.
