  Spiking neural networks (SNNs) offer a promising pathway to implement deep
neural networks (DNNs) in a more energy-efficient manner since their neurons
are sparsely activated and inferences are event-driven. However, there have
been very few works that have demonstrated the efficacy of SNNs in language
tasks partially because it is non-trivial to represent words in the forms of
spikes and to deal with variable-length texts by SNNs. This work presents a
"conversion + fine-tuning" two-step method for training SNNs for text
classification and proposes a simple but effective way to encode pre-trained
word embeddings as spike trains. We show empirically that after fine-tuning
with surrogate gradients, the converted SNNs achieve comparable results to
their DNN counterparts with much less energy consumption across multiple
datasets for both English and Chinese. We also show that such SNNs are more
robust to adversarial attacks than DNNs.
