  Spiking neural networks (SNNs) are well known as the brain-inspired models
with high computing efficiency, due to a key component that they utilize spikes
as information units, close to the biological neural systems. Although spiking
based models are energy efficient by taking advantage of discrete spike
signals, their performance is limited by current network structures and their
training methods. As discrete signals, typical SNNs cannot apply the gradient
descent rules directly into parameters adjustment as artificial neural networks
(ANNs). Aiming at this limitation, here we propose a novel method of
constructing deep SNN models with knowledge distillation (KD) that uses ANN as
teacher model and SNN as student model. Through ANN-SNN joint training
algorithm, the student SNN model can learn rich feature information from the
teacher ANN model through the KD method, yet it avoids training SNN from
scratch when communicating with non-differentiable spikes. Our method can not
only build a more efficient deep spiking structure feasibly and reasonably, but
use few time steps to train whole model compared to direct training or ANN to
SNN methods. More importantly, it has a superb ability of noise immunity for
various types of artificial noises and natural signals. The proposed novel
method provides efficient ways to improve the performance of SNN through
constructing deeper structures in a high-throughput fashion, with potential
usage for light and efficient brain-inspired computing of practical scenarios.
