  Training spiking neural networks to approximate complex functions is
essential for studying information processing in the brain and neuromorphic
computing. Yet, the binary nature of spikes constitutes a challenge for direct
gradient-based training. To sidestep this problem, surrogate gradients have
proven empirically successful, but their theoretical foundation remains
elusive. Here, we investigate the relation of surrogate gradients to two
theoretically well-founded approaches. On the one hand, we consider smoothed
probabilistic models, which, due to lack of support for automatic
differentiation, are impractical for training deep spiking neural networks, yet
provide gradients equivalent to surrogate gradients in single neurons. On the
other hand, we examine stochastic automatic differentiation, which is
compatible with discrete randomness but has never been applied to spiking
neural network training. We find that the latter provides the missing
theoretical basis for surrogate gradients in stochastic spiking neural
networks. We further show that surrogate gradients in deterministic networks
correspond to a particular asymptotic case and numerically confirm the
effectiveness of surrogate gradients in stochastic multi-layer spiking neural
networks. Finally, we illustrate that surrogate gradients are not conservative
fields and, thus, not gradients of a surrogate loss. Our work provides the
missing theoretical foundation for surrogate gradients and an analytically
well-founded solution for end-to-end training of stochastic spiking neural
networks.
