  The aim of the present paper is to study the effects of Hebbian learning in
random recurrent neural networks with biological connectivity, i.e. sparse
connections and separate populations of excitatory and inhibitory neurons. We
furthermore consider that the neuron dynamics may occur at a (shorter) time
scale than synaptic plasticity and consider the possibility of learning rules
with passive forgetting. We show that the application of such Hebbian learning
leads to drastic changes in the network dynamics and structure. In particular,
the learning rule contracts the norm of the weight matrix and yields a rapid
decay of the dynamics complexity and entropy. In other words, the network is
rewired by Hebbian learning into a new synaptic structure that emerges with
learning on the basis of the correlations that progressively build up between
neurons. We also observe that, within this emerging structure, the strongest
synapses organize as a small-world network. The second effect of the decay of
the weight matrix spectral radius consists in a rapid contraction of the
spectral radius of the Jacobian matrix. This drives the system through the
``edge of chaos'' where sensitivity to the input pattern is maximal. Taken
together, this scenario is remarkably predicted by theoretical arguments
derived from dynamical systems and graph theory.
