  In this paper, we investigate how to learn rich and robust feature
representations for audio classification from visual data and acoustic images,
a novel audio data modality. Former models learn audio representations from raw
signals or spectral data acquired by a single microphone, with remarkable
results in classification and retrieval. However, such representations are not
so robust towards variable environmental sound conditions. We tackle this
drawback by exploiting a new multimodal labeled action recognition dataset
acquired by a hybrid audio-visual sensor that provides RGB video, raw audio
signals, and spatialized acoustic data, also known as acoustic images, where
the visual and acoustic images are aligned in space and synchronized in time.
Using this richer information, we train audio deep learning models in a
teacher-student fashion. In particular, we distill knowledge into audio
networks from both visual and acoustic image teachers. Our experiments suggest
that the learned representations are more powerful and have better
generalization capabilities than the features learned from models trained using
just single-microphone audio data.
