  Spiking neural networks (SNNs) underlie low-power, fault-tolerant information
processing in the brain and could constitute a power-efficient alternative to
conventional deep neural networks when implemented on suitable neuromorphic
hardware accelerators. However, instantiating SNNs that solve complex
computational tasks in-silico remains a significant challenge. Surrogate
gradient (SG) techniques have emerged as a standard solution for training SNNs
end-to-end. Still, their success depends on synaptic weight initialization,
similar to conventional artificial neural networks (ANNs). Yet, unlike in the
case of ANNs, it remains elusive what constitutes a good initial state for an
SNN. Here, we develop a general initialization strategy for SNNs inspired by
the fluctuation-driven regime commonly observed in the brain. Specifically, we
derive practical solutions for data-dependent weight initialization that ensure
fluctuation-driven firing in the widely used leaky integrate-and-fire (LIF)
neurons. We empirically show that SNNs initialized following our strategy
exhibit superior learning performance when trained with SGs. These findings
generalize across several datasets and SNN architectures, including fully
connected, deep convolutional, recurrent, and more biologically plausible SNNs
obeying Dale's law. Thus fluctuation-driven initialization provides a
practical, versatile, and easy-to-implement strategy for improving SNN training
performance on diverse tasks in neuromorphic engineering and computational
neuroscience.
