  The synergy between spiking neural networks and neuromorphic hardware holds
promise for the development of energy-efficient AI applications. Inspired by
this potential, we revisit the foundational aspects to study the capabilities
of spiking neural networks where information is encoded in the firing time of
neurons. Under the Spike Response Model as a mathematical model of a spiking
neuron with a linear response function, we compare the expressive power of
artificial and spiking neural networks, where we initially show that they
realize piecewise linear mappings. In contrast to ReLU networks, we prove that
spiking neural networks can realize both continuous and discontinuous
functions. Moreover, we provide complexity bounds on the size of spiking neural
networks to emulate multi-layer (ReLU) neural networks. Restricting to the
continuous setting, we also establish complexity bounds in the reverse
direction for one-layer spiking neural networks.
