  Spiking neural networks (SNNs) have been gaining interest as energy-efficient
alternatives of conventional artificial neural networks (ANNs) due to their
event-driven computation. Considering the future deployment of SNN models to
constrained neuromorphic devices, many studies have applied techniques
originally used for ANN model compression, such as network quantization,
pruning, and knowledge distillation, to SNNs. Among them, existing works on
knowledge distillation reported accuracy improvements of student SNN model.
However, analysis on energy efficiency, which is also an important feature of
SNN, was absent. In this paper, we thoroughly analyze the performance of the
distilled SNN model in terms of accuracy and energy efficiency. In the process,
we observe a substantial increase in the number of spikes, leading to energy
inefficiency, when using the conventional knowledge distillation methods. Based
on this analysis, to achieve energy efficiency, we propose a novel knowledge
distillation method with heterogeneous temperature parameters. We evaluate our
method on two different datasets and show that the resulting SNN student
satisfies both accuracy improvement and reduction of the number of spikes. On
MNIST dataset, our proposed student SNN achieves up to 0.09% higher accuracy
and produces 65% less spikes compared to the student SNN trained with
conventional knowledge distillation method. We also compare the results with
other SNN compression techniques and training methods.
