  In this paper, we propose an effective sound event detection (SED) method
based on the audio spectrogram transformer (AST) model, pretrained on the
large-scale AudioSet for audio tagging (AT) task, termed AST-SED. Pretrained
AST models have recently shown promise on DCASE2022 challenge task4 where they
help mitigate a lack of sufficient real annotated data. However, mainly due to
differences between the AT and SED tasks, it is suboptimal to directly utilize
outputs from a pretrained AST model. Hence the proposed AST-SED adopts an
encoder-decoder architecture to enable effective and efficient fine-tuning
without needing to redesign or retrain the AST model. Specifically, the
Frequency-wise Transformer Encoder (FTE) consists of transformers with self
attention along the frequency axis to address multiple overlapped audio events
issue in a single clip. The Local Gated Recurrent Units Decoder (LGD) consists
of nearest-neighbor interpolation (NNI) and Bidirectional Gated Recurrent Units
(Bi-GRU) to compensate for temporal resolution loss in the pretrained AST model
output. Experimental results on DCASE2022 task4 development set have
demonstrated the superiority of the proposed AST-SED with FTE-LGD architecture.
Specifically, the Event-Based F1-score (EB-F1) of 59.60% and Polyphonic Sound
detection Score scenario1 (PSDS1) score of 0.5140 significantly outperform CRNN
and other pretrained AST-based systems.
