  Recent research has successfully adapted vision-based convolutional neural
network (CNN) architectures for audio recognition tasks using Mel-Spectrograms.
However, these CNNs have high computational costs and memory requirements,
limiting their deployment on low-end edge devices. Motivated by the success of
efficient vision models like InceptionNeXt and ConvNeXt, we propose
AudioRepInceptionNeXt, a single-stream architecture. Its basic building block
breaks down the parallel multi-branch depth-wise convolutions with descending
scales of k x k kernels into a cascade of two multi-branch depth-wise
convolutions. The first multi-branch consists of parallel multi-scale 1 x k
depth-wise convolutional layers followed by a similar multi-branch employing
parallel multi-scale k x 1 depth-wise convolutional layers. This reduces
computational and memory footprint while separating time and frequency
processing of Mel-Spectrograms. The large kernels capture global frequencies
and long activities, while small kernels get local frequencies and short
activities. We also reparameterize the multi-branch design during inference to
further boost speed without losing accuracy. Experiments show that
AudioRepInceptionNeXt reduces parameters and computations by 50%+ and improves
inference speed 1.28x over state-of-the-art CNNs like the Slow-Fast while
maintaining comparable accuracy. It also learns robustly across a variety of
audio recognition tasks. Codes are available at
https://github.com/StevenLauHKHK/AudioRepInceptionNeXt.
