  The brain is the perfect place to look for inspiration to develop more
efficient neural networks. The inner workings of our synapses and neurons
provide a glimpse at what the future of deep learning might look like. This
paper serves as a tutorial and perspective showing how to apply the lessons
learnt from several decades of research in deep learning, gradient descent,
backpropagation and neuroscience to biologically plausible spiking neural
neural networks.
  We also explore the delicate interplay between encoding data as spikes and
the learning process; the challenges and solutions of applying gradient-based
learning to spiking neural networks (SNNs); the subtle link between temporal
backpropagation and spike timing dependent plasticity, and how deep learning
might move towards biologically plausible online learning. Some ideas are well
accepted and commonly used amongst the neuromorphic engineering community,
while others are presented or justified for the first time here.
  The fields of deep learning and spiking neural networks evolve very rapidly.
We endeavour to treat this document as a 'dynamic' manuscript that will
continue to be updated as the common practices in training SNNs also change.
  A series of companion interactive tutorials complementary to this paper using
our Python package, snnTorch, are also made available. See
https://snntorch.readthedocs.io/en/latest/tutorials/index.html .
