  Spiking Neural Networks (SNNs) are a promising research direction for
building power-efficient information processing systems, especially for
temporal tasks such as speech recognition. In SNNs, delays refer to the time
needed for one spike to travel from one neuron to another. These delays matter
because they influence the spike arrival times, and it is well-known that
spiking neurons respond more strongly to coincident input spikes. More
formally, it has been shown theoretically that plastic delays greatly increase
the expressivity in SNNs. Yet, efficient algorithms to learn these delays have
been lacking. Here, we propose a new discrete-time algorithm that addresses
this issue in deep feedforward SNNs using backpropagation, in an offline
manner. To simulate delays between consecutive layers, we use 1D convolutions
across time. The kernels contain only a few non-zero weights - one per synapse
- whose positions correspond to the delays. These positions are learned
together with the weights using the recently proposed Dilated Convolution with
Learnable Spacings (DCLS). We evaluated our method on three datasets: the
Spiking Heidelberg Dataset (SHD), the Spiking Speech Commands (SSC) and its
non-spiking version Google Speech Commands v0.02 (GSC) benchmarks, which
require detecting temporal patterns. We used feedforward SNNs with two or three
hidden fully connected layers, and vanilla leaky integrate-and-fire neurons. We
showed that fixed random delays help and that learning them helps even more.
Furthermore, our method outperformed the state-of-the-art in the three datasets
without using recurrent connections and with substantially fewer parameters.
Our work demonstrates the potential of delay learning in developing accurate
and precise models for temporal data processing. Our code is based on PyTorch /
SpikingJelly and available at: https://github.com/Thvnvtos/SNN-delays
