  The Spiking Neural Network (SNN), as one of the biologically inspired neural
network infrastructures, has drawn increasing attention recently. It adopts
binary spike activations to transmit information, thus the multiplications of
activations and weights can be substituted by additions, which brings high
energy efficiency. However, in the paper, we theoretically and experimentally
prove that the binary spike activation map cannot carry enough information,
thus causing information loss and resulting in accuracy decreasing. To handle
the problem, we propose a ternary spike neuron to transmit information. The
ternary spike neuron can also enjoy the event-driven and multiplication-free
operation advantages of the binary spike neuron but will boost the information
capacity. Furthermore, we also embed a trainable factor in the ternary spike
neuron to learn the suitable spike amplitude, thus our SNN will adopt different
spike amplitudes along layers, which can better suit the phenomenon that the
membrane potential distributions are different along layers. To retain the
efficiency of the vanilla ternary spike, the trainable ternary spike SNN will
be converted to a standard one again via a re-parameterization technique in the
inference. Extensive experiments with several popular network structures over
static and dynamic datasets show that the ternary spike can consistently
outperform state-of-the-art methods. Our code is open-sourced at
https://github.com/yfguo91/Ternary-Spike.
