  It is widely appreciated that well-balanced excitation and inhibition are
necessary for proper function in neural networks. However, in principle, such
balance could be achieved by many possible configurations of excitatory and
inhibitory strengths, and relative numbers of excitatory and inhibitory
neurons. For instance, a given level of excitation could be balanced by either
numerous inhibitory neurons with weak synapses, or few inhibitory neurons with
strong synapses. Among the continuum of different but balanced configurations,
why should any particular configuration be favored? Here we address this
question in the context of the entropy of network dynamics by studying an
analytically tractable network of binary neurons. We find that entropy is
highest at the boundary between excitation-dominant and inhibition-dominant
regimes. Entropy also varies along this boundary with a trade-off between high
and robust entropy: weak synapse strengths yield high network entropy which is
fragile to parameter variations, while strong synapse strengths yield a lower,
but more robust, network entropy. In the case where inhibitory and excitatory
synapses are constrained to have similar strength, we find that a small, but
non-zero fraction of inhibitory neurons, like that seen in mammalian cortex,
results in robust and relatively high entropy.
