  Transient amplification has been proposed as an important mechanism not only
in neuroscience but in many areas modeled by dynamical systems. Despite that,
there is no clear biologically plausible mechanism which fine-tunes the
coupling matrix or selects signals to be amplified. In this work we
quantitatively study transient dynamics in the Rajan-Abbott model of a
recurrent neural network [K. Rajan and L.F. Abbot PRL 97, 188104 (2006)]. We
find a second order transition between a phase of weakly or no amplified
transients and a phase of strong amplification, where the average trajectory is
amplified. In the latter phase the combination of Dale's principle and
excitatory/inhibitory balance allows for strong weights, while maintaining the
system at the edge of chaos. Moreover, we show that the amplification goes hand
in hand with greater variability of the dynamics. By numerically studying the
full probability density of the squared norm, we observe as the strength of
weights grows, the right tail of the distribution becomes heavier, moving from
the Gaussian to the exponential tail.
