  Deep feedforward and recurrent rate-based neural networks have become
successful functional models of the brain, but they neglect obvious biological
details such as spikes and Dale's law. Here we argue that these details are
crucial in order to understand how real neural circuits operate. Towards this
aim, we put forth a new framework for spike-based computation in low-rank
excitatory-inhibitory spiking networks. By considering populations with rank-1
connectivity, we cast each neuron's spiking threshold as a boundary in a
low-dimensional input-output space. We then show how the combined thresholds of
a population of inhibitory neurons form a stable boundary in this space, and
those of a population of excitatory neurons form an unstable boundary.
Combining the two boundaries results in a rank-2 excitatory-inhibitory (EI)
network with inhibition-stabilized dynamics at the intersection of the two
boundaries. The computation of the resulting networks can be understood as the
difference of two convex functions and is thereby capable of approximating
arbitrary non-linear input-output mappings. We demonstrate several properties
of these networks, including noise suppression and amplification, irregular
activity and synaptic balance, as well as how they relate to rate network
dynamics in the limit that the boundary becomes soft. Finally, while our work
focuses on small networks (5-50 neurons), we discuss potential avenues for
scaling up to much larger networks. Overall, our work proposes a new
perspective on spiking networks that may serve as a starting point for a
mechanistic understanding of biological spike-based computation.
