  Drawing on the intricate structures of the brain, Spiking Neural Networks
(SNNs) emerge as a transformative development in artificial intelligence,
closely emulating the complex dynamics of biological neural networks. While
SNNs show promising efficiency on specialized sparse-computational hardware,
their practical training often relies on conventional GPUs. This reliance
frequently leads to extended computation times when contrasted with traditional
Artificial Neural Networks (ANNs), presenting significant hurdles for advancing
SNN research. To navigate this challenge, we present a novel temporal fusion
method, specifically designed to expedite the propagation dynamics of SNNs on
GPU platforms, which serves as an enhancement to the current significant
approaches for handling deep learning tasks with SNNs. This method underwent
thorough validation through extensive experiments in both authentic training
scenarios and idealized conditions, confirming its efficacy and adaptability
for single and multi-GPU systems. Benchmarked against various existing SNN
libraries/implementations, our method achieved accelerations ranging from
$5\times$ to $40\times$ on NVIDIA A100 GPUs. Publicly available experimental
codes can be found at https://github.com/EMI-Group/snn-temporal-fusion.
