  Spiking neural networks (SNNs) are promising brain-inspired energy-efficient
models. Compared to conventional deep Artificial Neural Networks (ANNs), SNNs
exhibit superior efficiency and capability to process temporal information.
However, it remains a challenge to train SNNs due to their undifferentiable
spiking mechanism. The surrogate gradients method is commonly used to train
SNNs, but often comes with an accuracy disadvantage over ANNs counterpart. We
link the degraded accuracy to the vanishing of gradient on the temporal
dimension through the analytical and experimental study of the training process
of Leaky Integrate-and-Fire (LIF) Neuron-based SNNs. Moreover, we propose the
Complementary Leaky Integrate-and-Fire (CLIF) Neuron. CLIF creates extra paths
to facilitate the backpropagation in computing temporal gradient while keeping
binary output. CLIF is hyperparameter-free and features broad applicability.
Extensive experiments on a variety of datasets demonstrate CLIF's clear
performance advantage over other neuron models. Furthermore, the CLIF's
performance even slightly surpasses superior ANNs with identical network
structure and training conditions. The code is available at
https://github.com/HuuYuLong/Complementary-LIF.
