  In this paper, we explore self-supervised audio-visual models that learn from
instructional videos. Prior work has shown that these models can relate spoken
words and sounds to visual content after training on a large-scale dataset of
videos, but they were only trained and evaluated on videos in English. To learn
multilingual audio-visual representations, we propose a cascaded approach that
leverages a model trained on English videos and applies it to audio-visual data
in other languages, such as Japanese videos. With our cascaded approach, we
show an improvement in retrieval performance of nearly 10x compared to training
on the Japanese videos solely. We also apply the model trained on English
videos to Japanese and Hindi spoken captions of images, achieving
state-of-the-art performance.
