  Sound classification models' performance suffers from generalizing on
out-of-distribution (OOD) data. Numerous methods have been proposed to help the
model generalize. However, most either introduce inference overheads or focus
on long-lasting CNN-variants, while Transformers has been proven to outperform
CNNs on numerous natural language processing and computer vision tasks. We
propose FRITO, an effective regularization technique on Transformer's
self-attention, to improve the model's generalization ability by limiting each
sequence position's attention receptive field along the frequency dimension on
the spectrogram. Experiments show that our method helps Transformer models
achieve SOTA generalization performance on TAU 2020 and Nsynth datasets while
saving 20% inference time.
