  The highly sparse activations in Spiking Neural Networks (SNNs) can provide
tremendous energy efficiency benefits when carefully exploited in hardware. The
behavior of sparsity in SNNs is uniquely shaped by the dataset and training
hyperparameters. This work reveals novel insights into the impacts of training
on hardware performance. Specifically, we explore the trade-offs between model
accuracy and hardware efficiency. We focus on three key hyperparameters:
surrogate gradient functions, beta, and membrane threshold. Results on an
FPGA-based hardware platform show that the fast sigmoid surrogate function
yields a lower firing rate with similar accuracy compared to the arctangent
surrogate on the SVHN dataset. Furthermore, by cross-sweeping the beta and
membrane threshold hyperparameters, we can achieve a 48% reduction in
hardware-based inference latency with only 2.88% trade-off in inference
accuracy compared to the default setting. Overall, this study highlights the
importance of fine-tuning model hyperparameters as crucial for designing
efficient SNN hardware accelerators, evidenced by the fine-tuned model
achieving a 1.72x improvement in accelerator efficiency (FPS/W) compared to the
most recent work.
