  Standard machine learning models for tagging and classifying acoustic signals
cannot handle classes that were not seen during training. Zero-Shot (ZS)
learning overcomes this restriction by predicting classes based on adaptable
class descriptions. This study sets out to investigate the effectiveness of
self-attention-based audio embedding architectures for ZS learning. To this
end, we compare the very recent patchout spectrogram transformer with two
classic convolutional architectures. We evaluate these three architectures on
three tasks and on three different benchmark datasets: general-purpose tagging
on AudioSet, environmental sound classification on ESC-50, and instrument
tagging on OpenMIC. Our results show that the self-attention-based embedding
methods outperform both compared convolutional architectures in all of these
settings. By designing training and test data accordingly, we observe that
prediction performance suffers significantly when the `semantic distance'
between training and new test classes is large, an effect that will deserve
more detailed investigations.
