  Spiking neural networks (SNNs), that operate via binary spikes distributed
over time, have emerged as a promising energy efficient ML paradigm for
resource-constrained devices. However, the current state-of-the-art (SOTA) SNNs
require multiple time steps for acceptable inference accuracy, increasing
spiking activity and, consequently, energy consumption. SOTA training
strategies for SNNs involve conversion from a non-spiking deep neural network
(DNN). In this paper, we determine that SOTA conversion strategies cannot yield
ultra low latency because they incorrectly assume that the DNN and SNN
pre-activation values are uniformly distributed. We propose a new training
algorithm that accurately captures these distributions, minimizing the error
between the DNN and converted SNN. The resulting SNNs have ultra low latency
and high activation sparsity, yielding significant improvements in compute
efficiency. In particular, we evaluate our framework on image recognition tasks
from CIFAR-10 and CIFAR-100 datasets on several VGG and ResNet architectures.
We obtain top-1 accuracy of 64.19% with only 2 time steps on the CIFAR-100
dataset with ~159.2x lower compute energy compared to an iso-architecture
standard DNN. Compared to other SOTA SNN models, our models perform inference
2.5-8x faster (i.e., with fewer time steps).
