  Current state-of-the-art audio analysis systems rely on pre-trained embedding
models, often used off-the-shelf as (frozen) feature extractors. Choosing the
best one for a set of tasks is the subject of many recent publications.
However, one aspect often overlooked in these works is the influence of the
duration of audio input considered to extract an embedding, which we refer to
as Temporal Support (TS). In this work, we study the influence of the TS for
well-established or emerging pre-trained embeddings, chosen to represent
different types of architectures and learning paradigms. We conduct this
evaluation using both musical instrument and environmental sound datasets,
namely OpenMIC, TAU Urban Acoustic Scenes 2020 Mobile, and ESC-50. We
especially highlight that Audio Spectrogram Transformer-based systems (PaSST
and BEATs) remain effective with smaller TS, which therefore allows for a
drastic reduction in memory and computational cost. Moreover, we show that by
choosing the optimal TS we reach competitive results across all tasks. In
particular, we improve the state-of-the-art results on OpenMIC, using BEATs and
PaSST without any fine-tuning.
