  Over the past few years, audio classification task on large-scale dataset
such as AudioSet has been an important research area. Several deeper
Convolution-based Neural networks have shown compelling performance notably
Vggish, YAMNet, and Pretrained Audio Neural Network (PANN). These models are
available as pretrained architecture for transfer learning as well as specific
audio task adoption. In this paper, we propose a lightweight on-device deep
learning-based model for audio classification, LEAN. LEAN consists of a raw
waveform-based temporal feature extractor called as Wave Encoder and
logmel-based Pretrained YAMNet. We show that using a combination of trainable
wave encoder, Pretrained YAMNet along with cross attention-based temporal
realignment, results in competitive performance on downstream audio
classification tasks with lesser memory footprints and hence making it suitable
for resource constraints devices such as mobile, edge devices, etc . Our
proposed system achieves on-device mean average precision(mAP) of .445 with a
memory footprint of a mere 4.5MB on the FSD50K dataset which is an improvement
of 22% over baseline on-device mAP on same dataset.
