  In neural circuits, synaptic strengths influence neuronal activity by shaping
network dynamics, and neuronal activity influences synaptic strengths through
activity-dependent plasticity. Motivated by this fact, we study a
recurrent-network model in which neuronal units and synaptic couplings are
interacting dynamic variables, with couplings subject to Hebbian modification
with decay around quenched random strengths. Rather than assigning a specific
role to the plasticity, we use dynamical mean-field theory and other techniques
to systematically characterize the neuronal-synaptic dynamics, revealing a rich
phase diagram. Adding Hebbian plasticity slows activity in chaotic networks and
can induce chaos in otherwise quiescent networks. Anti-Hebbian plasticity
quickens activity and produces an oscillatory component. Analysis of the
Jacobian shows that Hebbian and anti-Hebbian plasticity push locally unstable
modes toward the real and imaginary axes, explaining these behaviors. Both
random-matrix and Lyapunov analysis show that strong Hebbian plasticity
segregates network timescales into two bands with a slow, synapse-dominated
band driving the dynamics, suggesting a flipped view of the network as synapses
connected by neurons. For increasing strength, Hebbian plasticity initially
raises the complexity of the dynamics, measured by the maximum Lyapunov
exponent and attractor dimension, but then decreases these metrics, likely due
to the proliferation of stable fixed points. We compute the marginally stable
spectra of such fixed points as well as their number, showing exponential
growth with network size. In chaotic states with strong Hebbian plasticity, a
stable fixed point of neuronal dynamics is destabilized by synaptic dynamics,
allowing any neuronal state to be stored as a stable fixed point by halting the
plasticity. This phase of freezable chaos offers a new mechanism for working
memory.
