  Existing audio analysis methods generally first transform the audio stream to
spectrogram, and then feed it into CNN for further analysis. A standard CNN
recognizes specific visual patterns over feature map, then pools for high-level
representation, which overlooks the positional information of recognized
patterns. However, unlike natural image, the semantic of an audio spectrogram
is sensitive to positional change, as its vertical and horizontal axes indicate
the frequency and temporal information of the audio, instead of naive
rectangular coordinates. Thus, the insensitivity of CNN to positional change
plays a negative role on audio spectrogram encoding. To address this issue,
this paper proposes a new self-supervised learning mechanism, which enhances
the audio representation by first generating adversarial samples
(\textit{i.e.}, negative samples), then driving CNN to distinguish the
embeddings of negative pairs in the latent space. Extensive experiments show
that the proposed approach achieves best or competitive results on 9 downstream
datasets compared with previous methods, which verifies its effectiveness on
audio representation learning.
