  The increasing need for compact and low-power computing solutions for machine
learning applications has triggered significant interest in energy-efficient
neuromorphic systems. However, most of these architectures rely on spiking
neural networks, which typically perform poorly compared to their non-spiking
counterparts in terms of accuracy. In this paper, we propose a new adaptive
spiking neuron model that can be abstracted as a low-pass filter. This
abstraction enables faster and better training of spiking networks using
back-propagation, without simulating spikes. We show that this model
dramatically improves the inference performance of a recurrent neural network
and validate it with three complex spatio-temporal learning tasks: the temporal
addition task, the temporal copying task, and a spoken-phrase recognition task.
We estimate at least 500x higher energy-efficiency using our models on
compatible neuromorphic chips in comparison to Cortex-M4, a popular embedded
microprocessor.
