  Traditional mathematical approaches to studying analytically the dynamics of
neural networks rely on the mean-field approximation, which is rigorously
applicable only to networks of infinite size. However, all existing real
biological networks have finite size, and many of them, such as microscopic
circuits in invertebrates, are composed only of a few tens of neurons. Thus, it
is important to be able to extend to small-size networks our ability to study
analytically neural dynamics. Analytical solutions of the dynamics of
finite-size neural networks have remained elusive for many decades, because the
powerful methods of statistical analysis, such as the central limit theorem and
the law of large numbers, do not apply to small networks. In this article, we
critically review recent progress on the study of the dynamics of small
networks composed of binary neurons. In particular, we review the mathematical
techniques we developed for studying the bifurcations of the network dynamics,
the dualism between neural activity and membrane potentials, cross-neuron
correlations, and pattern storage in stochastic networks. Finally, we highlight
key challenges that remain open, future directions for further progress, and
possible implications of our results for neuroscience.
