  Among the array of neural network architectures, the Vision Transformer (ViT)
stands out as a prominent choice, acclaimed for its exceptional expressiveness
and consistent high performance in various vision applications. Recently, the
emerging Spiking ViT approach has endeavored to harness spiking neurons, paving
the way for a more brain-inspired transformer architecture that thrives in
ultra-low power operations on dedicated neuromorphic hardware. Nevertheless,
this approach remains confined to spatial self-attention and doesn't fully
unlock the potential of spiking neural networks. We introduce DISTA, a
Denoising Spiking Transformer with Intrinsic Plasticity and SpatioTemporal
Attention, designed to maximize the spatiotemporal computational prowess of
spiking neurons, particularly for vision applications. DISTA explores two types
of spatiotemporal attentions: intrinsic neuron-level attention and
network-level attention with explicit memory. Additionally, DISTA incorporates
an efficient nonlinear denoising mechanism to quell the noise inherent in
computed spatiotemporal attention maps, thereby resulting in further
performance gains. Our DISTA transformer undergoes joint training involving
synaptic plasticity (i.e., weight tuning) and intrinsic plasticity (i.e.,
membrane time constant tuning) and delivers state-of-the-art performances
across several static image and dynamic neuromorphic datasets. With only 6 time
steps, DISTA achieves remarkable top-1 accuracy on CIFAR10 (96.26%) and
CIFAR100 (79.15%), as well as 79.1% on CIFAR10-DVS using 10 time steps.
