  The training of stochastic neural network models with binary ($\pm1$) weights
and activations via continuous surrogate networks is investigated. We derive
new surrogates using a novel derivation based on writing the stochastic neural
network as a Markov chain. This derivation also encompasses existing variants
of the surrogates presented in the literature. Following this, we theoretically
study the surrogates at initialisation. We derive, using mean field theory, a
set of scalar equations describing how input signals propagate through the
randomly initialised networks. The equations reveal whether so-called critical
initialisations exist for each surrogate network, where the network can be
trained to arbitrary depth. Moreover, we predict theoretically and confirm
numerically, that common weight initialisation schemes used in standard
continuous networks, when applied to the mean values of the stochastic binary
weights, yield poor training performance. This study shows that, contrary to
common intuition, the means of the stochastic binary weights should be
initialised close to $\pm 1$, for deeper networks to be trainable.
