  In this note, we develop semi-analytical techniques to obtain the full
correlational structure of a stochastic network of nonlinear neurons described
by rate variables. Under the assumption that pairs of membrane potentials are
jointly Gaussian -- which they tend to be in large networks -- we obtain
deterministic equations for the temporal evolution of the mean firing rates and
the noise covariance matrix that can be solved straightforwardly given the
network connectivity. We also obtain spike count statistics such as Fano
factors and pairwise correlations, assuming doubly-stochastic action potential
firing. Importantly, our theory does not require fluctuations to be small, and
works for several biologically motivated, convex single-neuron nonlinearities.
