  In computer vision, convolutional neural networks (CNN) such as ConvNeXt,
have been able to surpass state-of-the-art transformers, partly thanks to
depthwise separable convolutions (DSC). DSC, as an approximation of the regular
convolution, has made CNNs more efficient in time and memory complexity without
deteriorating their accuracy, and sometimes even improving it. In this paper,
we first implement DSC into the Pretrained Audio Neural Networks (PANN) family
for audio classification on AudioSet, to show its benefits in terms of
accuracy/model size trade-off. Second, we adapt the now famous ConvNeXt model
to the same task. It rapidly overfits, so we report on techniques that improve
the learning process. Our best ConvNeXt model reached 0.471 mean-average
precision on AudioSet, which is better than or equivalent to recent large audio
transformers, while using three times less parameters. We also achieved
positive results in audio captioning and audio retrieval with this model. Our
PyTorch source code and checkpoint models are available at
https://github.com/topel/audioset-convnext-inf.
