  Spiking Neural Networks (SNNs) are attracting widespread interest due to
their biological plausibility, energy efficiency, and powerful spatio-temporal
information representation ability. Given the critical role of attention
mechanisms in enhancing neural network performance, the integration of SNNs and
attention mechanisms exhibits potential to deliver energy-efficient and
high-performance computing paradigms. We present a novel Temporal-Channel Joint
Attention mechanism for SNNs, referred to as TCJA-SNN. The proposed TCJA-SNN
framework can effectively assess the significance of spike sequence from both
spatial and temporal dimensions. More specifically, our essential technical
contribution lies on: 1) We employ the squeeze operation to compress the spike
stream into an average matrix. Then, we leverage two local attention mechanisms
based on efficient 1D convolutions to facilitate comprehensive feature
extraction at the temporal and channel levels independently. 2) We introduce
the Cross Convolutional Fusion (CCF) layer as a novel approach to model the
inter-dependencies between the temporal and channel scopes. This layer breaks
the independence of these two dimensions and enables the interaction between
features. Experimental results demonstrate that the proposed TCJA-SNN
outperforms SOTA by up to 15.7% accuracy on standard static and neuromorphic
datasets, including Fashion-MNIST, CIFAR10-DVS, N-Caltech 101, and DVS128
Gesture. Furthermore, we apply the TCJA-SNN framework to image generation tasks
by leveraging a variation autoencoder. To the best of our knowledge, this study
is the first instance where the SNN-attention mechanism has been employed for
image classification and generation tasks. Notably, our approach has achieved
SOTA performance in both domains, establishing a significant advancement in the
field. Codes are available at https://github.com/ridgerchu/TCJA.
