  Birds are vital parts of ecosystems across the world and are an excellent
measure of the quality of life on earth. Many bird species are endangered while
others are already extinct. Ecological efforts in understanding and monitoring
bird populations are important to conserve their habitat and species, but this
mostly relies on manual methods in rough terrains. Recent advances in Machine
Learning and Deep Learning have made automatic bird recognition in diverse
environments possible. Birdcall recognition till now has been performed using
convolutional neural networks. In this work, we try and understand how
self-attention can aid in this endeavor. With that we build an pre-trained
Attention-based Spectrogram Transformer baseline for BirdCLEF 2022 and compare
the results against the pre-trained Convolution-based baseline. Our results
show that the transformer models outperformed the convolutional model and we
further validate our results by building baselines and analyzing the results
for the previous year BirdCLEF 2021 challenge. Source code available at
https://github.com/ck090/BirdCLEF-22
