  Bioacoustic research, vital for understanding animal behavior, conservation,
and ecology, faces a monumental challenge: analyzing vast datasets where animal
vocalizations are rare. While deep learning techniques are becoming standard,
adapting them to bioacoustics remains difficult. We address this with
animal2vec, an interpretable large transformer model, and a self-supervised
training scheme tailored for sparse and unbalanced bioacoustic data. It learns
from unlabeled audio and then refines its understanding with labeled data.
Furthermore, we introduce and publicly release MeerKAT: Meerkat Kalahari Audio
Transcripts, a dataset of meerkat (Suricata suricatta) vocalizations with
millisecond-resolution annotations, the largest labeled dataset on non-human
terrestrial mammals currently available. Our model outperforms existing methods
on MeerKAT and the publicly available NIPS4Bplus birdsong dataset. Moreover,
animal2vec performs well even with limited labeled data (few-shot learning).
animal2vec and MeerKAT provide a new reference point for bioacoustic research,
enabling scientists to analyze large amounts of data even with scarce ground
truth information.
