  Despite the rapid progress of neuromorphic computing, the inadequate depth
and the resulting insufficient representation power of spiking neural networks
(SNNs) severely restrict their application scope in practice. Residual learning
and shortcuts have been evidenced as an important approach for training deep
neural networks, but rarely did previous work assess their applicability to the
characteristics of spike-based communication and spatiotemporal dynamics. This
negligence leads to impeded information flow and the accompanying degradation
problem. In this paper, we identify the crux and then propose a novel residual
block for SNNs, which is able to significantly extend the depth of directly
trained SNNs, e.g., up to 482 layers on CIFAR-10 and 104 layers on ImageNet,
without observing any slight degradation problem. We validate the effectiveness
of our methods on both frame-based and neuromorphic datasets, and our
SRM-ResNet104 achieves a superior result of 76.02% accuracy on ImageNet, the
first time in the domain of directly trained SNNs. The great energy efficiency
is estimated and the resulting networks need on average only one spike per
neuron for classifying an input sample. We believe our powerful and scalable
modeling will provide a strong support for further exploration of SNNs.
