  Self-supervised learning (SSL) has recently shown remarkable results in
closing the gap between supervised and unsupervised learning. The idea is to
learn robust features that are invariant to distortions of the input data.
Despite its success, this idea can suffer from a collapsing issue where the
network produces a constant representation. To this end, we introduce SELFIE, a
novel Self-supervised Learning approach for audio representation via Feature
Diversity and Decorrelation. SELFIE avoids the collapsing issue by ensuring
that the representation (i) maintains a high diversity among embeddings and
(ii) decorrelates the dependencies between dimensions. SELFIE is pre-trained on
the large-scale AudioSet dataset and its embeddings are validated on nine audio
downstream tasks, including speech, music, and sound event recognition.
Experimental results show that SELFIE outperforms existing SSL methods in
several tasks.
