  Spiking Neural Networks (SNNs) have emerged as an attractive spatio-temporal
computing paradigm for complex vision tasks. However, most existing works yield
models that require many time steps and do not leverage the inherent temporal
dynamics of spiking neural networks, even for sequential tasks. Motivated by
this observation, we propose an \rev{optimized spiking long short-term memory
networks (LSTM) training framework that involves a novel ANN-to-SNN conversion
framework, followed by SNN training}. In particular, we propose novel
activation functions in the source LSTM architecture and judiciously select a
subset of them for conversion to integrate-and-fire (IF) activations with
optimal bias shifts. Additionally, we derive the leaky-integrate-and-fire (LIF)
activation functions converted from their non-spiking LSTM counterparts which
justifies the need to jointly optimize the weights, threshold, and leak
parameter. We also propose a pipelined parallel processing scheme which hides
the SNN time steps, significantly improving system latency, especially for long
sequences. The resulting SNNs have high activation sparsity and require only
accumulate operations (AC), in contrast to expensive multiply-and-accumulates
(MAC) needed for ANNs, except for the input layer when using direct encoding,
yielding significant improvements in energy efficiency. We evaluate our
framework on sequential learning tasks including temporal MNIST, Google Speech
Commands (GSC), and UCI Smartphone datasets on different LSTM architectures. We
obtain test accuracy of 94.75% with only 2 time steps with direct encoding on
the GSC dataset with 4.1x lower energy than an iso-architecture standard LSTM.
