  Deep spiking neural networks (SNNs) offer the promise of low-power artificial
intelligence. However, training deep SNNs from scratch or converting deep
artificial neural networks to SNNs without loss of performance has been a
challenge. Here we propose an exact mapping from a network with Rectified
Linear Units (ReLUs) to an SNN that fires exactly one spike per neuron. For our
constructive proof, we assume that an arbitrary multi-layer ReLU network with
or without convolutional layers, batch normalization and max pooling layers was
trained to high performance on some training set. Furthermore, we assume that
we have access to a representative example of input data used during training
and to the exact parameters (weights and biases) of the trained ReLU network.
The mapping from deep ReLU networks to SNNs causes zero percent drop in
accuracy on CIFAR10, CIFAR100 and the ImageNet-like data sets Places365 and
PASS. More generally our work shows that an arbitrary deep ReLU network can be
replaced by an energy-efficient single-spike neural network without any loss of
performance.
