  A fundamental feature of complex biological systems is the ability to form
feedback interactions with their environment. A prominent model for studying
such interactions is reservoir computing, where learning acts on
low-dimensional bottlenecks. Despite the simplicity of this learning scheme,
the factors contributing to or hindering the success of training in reservoir
networks are in general not well understood. In this work, we study non-linear
feedback networks trained to generate a sinusoidal signal, and analyze how
learning performance is shaped by the interplay between internal network
dynamics and target properties. By performing exact mathematical analysis of
linearized networks, we predict that learning performance is maximized when the
target is characterized by an optimal, intermediate frequency which
monotonically decreases with the strength of the internal reservoir
connectivity. At the optimal frequency, the reservoir representation of the
target signal is high-dimensional, de-synchronized, and thus maximally robust
to noise. We show that our predictions successfully capture the qualitative
behaviour of performance in non-linear networks. Moreover, we find that the
relationship between internal representations and performance can be further
exploited in trained non-linear networks to explain behaviours which do not
have a linear counterpart. Our results indicate that a major determinant of
learning success is the quality of the internal representation of the target,
which in turn is shaped by an interplay between parameters controlling the
internal network and those defining the task.
