  Spiking Neural Networks (SNNs) are promising energy-efficient models for
neuromorphic computing. For training the non-differentiable SNN models, the
backpropagation through time (BPTT) with surrogate gradients (SG) method has
achieved high performance. However, this method suffers from considerable
memory cost and training time during training. In this paper, we propose the
Spatial Learning Through Time (SLTT) method that can achieve high performance
while greatly improving training efficiency compared with BPTT. First, we show
that the backpropagation of SNNs through the temporal domain contributes just a
little to the final calculated gradients. Thus, we propose to ignore the
unimportant routes in the computational graph during backpropagation. The
proposed method reduces the number of scalar multiplications and achieves a
small memory occupation that is independent of the total time steps.
Furthermore, we propose a variant of SLTT, called SLTT-K, that allows
backpropagation only at K time steps, then the required number of scalar
multiplications is further reduced and is independent of the total time steps.
Experiments on both static and neuromorphic datasets demonstrate superior
training efficiency and performance of our SLTT. In particular, our method
achieves state-of-the-art accuracy on ImageNet, while the memory cost and
training time are reduced by more than 70% and 50%, respectively, compared with
BPTT.
