  Audio classification is vital in areas such as speech and music recognition.
Feature extraction from the audio signal, such as Mel-Spectrograms and MFCCs,
is a critical step in audio classification. These features are transformed into
spectrograms for classification. Researchers have explored various techniques,
including traditional machine and deep learning methods to classify
spectrograms, but these can be computationally expensive. To simplify this
process, a more straightforward approach inspired by sequence classification in
NLP can be used. This paper proposes a Transformer-encoder-based model for
audio classification using MFCCs. The model was benchmarked against the ESC-50,
Speech Commands v0.02 and UrbanSound8k datasets and has shown strong
performance, with the highest accuracy of 95.2% obtained upon training the
model on the UrbanSound8k dataset. The model consisted of a mere 127,544 total
parameters, making it light-weight yet highly efficient at the audio
classification task.
