  Spiking neural networks (SNNs) with event-based computation are promising
brain-inspired models for energy-efficient applications on neuromorphic
hardware. However, most supervised SNN training methods, such as conversion
from artificial neural networks or direct training with surrogate gradients,
require complex computation rather than spike-based operations of spiking
neurons during training. In this paper, we study spike-based implicit
differentiation on the equilibrium state (SPIDE) that extends the recently
proposed training method, implicit differentiation on the equilibrium state
(IDE), for supervised learning with purely spike-based computation, which
demonstrates the potential for energy-efficient training of SNNs. Specifically,
we introduce ternary spiking neuron couples and prove that implicit
differentiation can be solved by spikes based on this design, so the whole
training procedure, including both forward and backward passes, is made as
event-driven spike computation, and weights are updated locally with two-stage
average firing rates. Then we propose to modify the reset membrane potential to
reduce the approximation error of spikes. With these key components, we can
train SNNs with flexible structures in a small number of time steps and with
firing sparsity during training, and the theoretical estimation of energy costs
demonstrates the potential for high efficiency. Meanwhile, experiments show
that even with these constraints, our trained models can still achieve
competitive results on MNIST, CIFAR-10, CIFAR-100, and CIFAR10-DVS. Our code is
available at https://github.com/pkuxmq/SPIDE-FSNN.
