  The tremendous energy consumption of deep neural networks (DNNs) has become a
serious problem in deep learning. Spiking neural networks (SNNs), which mimic
the operations in the human brain, have been studied as prominent
energy-efficient neural networks. Due to their event-driven and
spatiotemporally sparse operations, SNNs show possibilities for
energy-efficient processing. To unlock their potential, deep SNNs have adopted
temporal coding such as time-to-first-spike (TTFS)coding, which represents the
information between neurons by the first spike time. With TTFS coding, each
neuron generates one spike at most, which leads to a significant improvement in
energy efficiency. Several studies have successfully introduced TTFS coding in
deep SNNs, but they showed restricted efficiency improvement owing to the lack
of consideration for efficiency during training. To address the aforementioned
issue, this paper presents training methods for energy-efficient deep SNNs with
TTFS coding. We introduce a surrogate DNN model to train the deep SNN in a
feasible time and analyze the effect of the temporal kernel on training
performance and efficiency. Based on the investigation, we propose
stochastically relaxed activation and initial value-based regularization for
the temporal kernel parameters. In addition, to reduce the number of spikes
even further, we present temporal kernel-aware batch normalization. With the
proposed methods, we could achieve comparable training results with
significantly reduced spikes, which could lead to energy-efficient deep SNNs.
