  Can replay, as a widely observed neural activity pattern in brain regions,
particularly in the hippocampus and neocortex, emerge in an artificial agent?
If yes, does it contribute to the tasks? In this work, without heavy dependence
on complex assumptions, we discover naturally emergent replay under
task-optimized paradigm using a recurrent neural network-based reinforcement
learning model, which mimics the hippocampus and prefrontal cortex, as well as
their intercommunication and the sensory cortex input. The emergent replay in
the hippocampus, which results from the episodic memory and cognitive map as
well as environment observations, well resembles animal experimental data and
serves as an effective indicator of high task performance. The model also
successfully reproduces local and nonlocal replay, which matches the human
experimental data. Our work provides a new avenue for understanding the
mechanisms behind replay.
