  This paper shows that the heterogeneity in neuronal and synaptic dynamics
reduces the spiking activity of a Recurrent Spiking Neural Network (RSNN) while
improving prediction performance, enabling spike-efficient (unsupervised)
learning. We analytically show that the diversity in neurons'
integration/relaxation dynamics improves an RSNN's ability to learn more
distinct input patterns (higher memory capacity), leading to improved
classification and prediction performance. We further prove that heterogeneous
Spike-Timing-Dependent-Plasticity (STDP) dynamics of synapses reduce spiking
activity but preserve memory capacity. The analytical results motivate
Heterogeneous RSNN design using Bayesian optimization to determine
heterogeneity in neurons and synapses to improve $\mathcal{E}$, defined as the
ratio of spiking activity and memory capacity. The empirical results on time
series classification and prediction tasks show that optimized HRSNN increases
performance and reduces spiking activity compared to a homogeneous RSNN.
