  Neural systems are well known for their ability to learn and store
information as memories. Even more impressive is their ability to abstract
these memories to create complex internal representations, enabling advanced
functions such as the spatial manipulation of mental representations. While
recurrent neural networks (RNNs) are capable of representing complex
information, the exact mechanisms of how dynamical neural systems perform
abstraction are still not well-understood, thereby hindering the development of
more advanced functions. Here, we train a 1000-neuron RNN -- a reservoir
computer (RC) -- to abstract a continuous dynamical attractor memory from
isolated examples of dynamical attractor memories. Further, we explain the
abstraction mechanism with new theory. By training the RC on isolated and
shifted examples of either stable limit cycles or chaotic Lorenz attractors,
the RC learns a continuum of attractors, as quantified by an extra Lyapunov
exponent equal to zero. We propose a theoretical mechanism of this abstraction
by combining ideas from differentiable generalized synchronization and feedback
dynamics. Our results quantify abstraction in simple neural systems, enabling
us to design artificial RNNs for abstraction, and leading us towards a neural
basis of abstraction.
