  Parameter-efficient transfer learning (PETL) methods have emerged as a solid
alternative to the standard full fine-tuning approach. They only train a few
extra parameters for each downstream task, without sacrificing performance and
dispensing with the issue of storing a copy of the pre-trained model for each
task. For audio classification tasks, the Audio Spectrogram Transformer (AST)
model shows impressive results. However, surprisingly, how to efficiently adapt
it to several downstream tasks has not been tackled before. In this paper, we
bridge this gap and present a detailed investigation of common PETL methods for
the adaptation of the AST model to audio/speech tasks. Furthermore, we propose
a new adapter design that exploits the convolution module of the Conformer
model, leading to superior performance over the standard PETL approaches and
surpassing or achieving performance parity with full fine-tuning by updating
only 0.29% of the parameters. Finally, we provide ablation studies revealing
that our proposed adapter: 1) proves to be effective in few-shot efficient
transfer learning, 2) attains optimal results regardless of the amount of the
allocated parameters, and 3) can be applied to other pre-trained models.
