  This paper presents a new methodology to alleviate the fundamental trade-off
between accuracy and latency in spiking neural networks (SNNs). The approach
involves decoding confidence information over time from the SNN outputs and
using it to develop a decision-making agent that can dynamically determine when
to terminate each inference.
  The proposed method, Dynamic Confidence, provides several significant
benefits to SNNs. 1. It can effectively optimize latency dynamically at
runtime, setting it apart from many existing low-latency SNN algorithms. Our
experiments on CIFAR-10 and ImageNet datasets have demonstrated an average 40%
speedup across eight different settings after applying Dynamic Confidence. 2.
The decision-making agent in Dynamic Confidence is straightforward to construct
and highly robust in parameter space, making it extremely easy to implement. 3.
The proposed method enables visualizing the potential of any given SNN, which
sets a target for current SNNs to approach. For instance, if an SNN can
terminate at the most appropriate time point for each input sample, a ResNet-50
SNN can achieve an accuracy as high as 82.47% on ImageNet within just 4.71 time
steps on average. Unlocking the potential of SNNs needs a highly-reliable
decision-making agent to be constructed and fed with a high-quality estimation
of ground truth. In this regard, Dynamic Confidence represents a meaningful
step toward realizing the potential of SNNs.
