  Spiking Transformers have gained considerable attention because they achieve
both the energy efficiency of Spiking Neural Networks (SNNs) and the high
capacity of Transformers. However, the existing Spiking Transformer
architectures, derived from Artificial Neural Networks (ANNs), exhibit a
notable architectural gap, resulting in suboptimal performance compared to
their ANN counterparts. Manually discovering optimal architectures is
time-consuming. To address these limitations, we introduce AutoST, a
training-free NAS method for Spiking Transformers, to rapidly identify
high-performance Spiking Transformer architectures. Unlike existing
training-free NAS methods, which struggle with the non-differentiability and
high sparsity inherent in SNNs, we propose to utilize Floating-Point Operations
(FLOPs) as a performance metric, which is independent of model computations and
training dynamics, leading to a stronger correlation with performance. Our
extensive experiments show that AutoST models outperform state-of-the-art
manually or automatically designed SNN architectures on static and neuromorphic
datasets. Full code, model, and data are released for reproduction.
