  Spiking neural networks (SNNs) that enables energy efficient implementation
on emerging neuromorphic hardware are gaining more attention. Yet now, SNNs
have not shown competitive performance compared with artificial neural networks
(ANNs), due to the lack of effective learning algorithms and efficient
programming frameworks. We address this issue from two aspects: (1) We propose
a neuron normalization technique to adjust the neural selectivity and develop a
direct learning algorithm for deep SNNs. (2) Via narrowing the rate coding
window and converting the leaky integrate-and-fire (LIF) model into an
explicitly iterative version, we present a Pytorch-based implementation method
towards the training of large-scale SNNs. In this way, we are able to train
deep SNNs with tens of times speedup. As a result, we achieve significantly
better accuracy than the reported works on neuromorphic datasets (N-MNIST and
DVS-CIFAR10), and comparable accuracy as existing ANNs and pre-trained SNNs on
non-spiking datasets (CIFAR10). {To our best knowledge, this is the first work
that demonstrates direct training of deep SNNs with high performance on
CIFAR10, and the efficient implementation provides a new way to explore the
potential of SNNs.
