  The ability to store and manipulate information is a hallmark of
computational systems. Whereas computers are carefully engineered to represent
and perform mathematical operations on structured data, neurobiological systems
perform analogous functions despite flexible organization and unstructured
sensory input. Recent efforts have made progress in modeling the representation
and recall of information in neural systems. However, precisely how neural
systems learn to modify these representations remains far from understood. Here
we demonstrate that a recurrent neural network (RNN) can learn to modify its
representation of complex information using only examples, and we explain the
associated learning mechanism with new theory. Specifically, we drive an RNN
with examples of translated, linearly transformed, or pre-bifurcated time
series from a chaotic Lorenz system, alongside an additional control signal
that changes value for each example. By training the network to replicate the
Lorenz inputs, it learns to autonomously evolve about a Lorenz-shaped manifold.
Additionally, it learns to continuously interpolate and extrapolate the
translation, transformation, and bifurcation of this representation far beyond
the training data by changing the control signal. Finally, we provide a
mechanism for how these computations are learned, and demonstrate that a single
network can simultaneously learn multiple computations. Together, our results
provide a simple but powerful mechanism by which an RNN can learn to manipulate
internal representations of complex information, allowing for the principled
study and precise design of RNNs.
