  Can neural networks learn goal-directed behaviour using similar strategies to
the brain, by combining the relationships between the current state of the
organism and the consequences of future actions? Recent work has shown that
recurrent neural networks trained on goal based tasks can develop
representations resembling those found in the brain, entorhinal cortex grid
cells, for instance. Here we explore the evolution of the dynamics of their
internal representations and compare this with experimental data. We observe
that once a recurrent network is trained to learn the structure of its
environment solely based on sensory prediction, an attractor based landscape
forms in the network's representation, which parallels hippocampal place cells
in structure and function. Next, we extend the predictive objective to include
Q-learning for a reward task, where rewarding actions are dependent on delayed
cue modulation. Mirroring experimental findings in hippocampus recordings in
rodents performing the same task, this training paradigm causes nonlocal neural
activity to sweep forward in space at decision points, anticipating the future
path to a rewarded location. Moreover, prevalent choice and cue-selective
neurons form in this network, again recapitulating experimental findings.
Together, these results indicate that combining predictive, unsupervised
learning of the structure of an environment with reinforcement learning can
help understand the formation of hippocampus-like representations containing
both spatial and task-relevant information.
