  We present SpeechMatrix, a large-scale multilingual corpus of
speech-to-speech translations mined from real speech of European Parliament
recordings. It contains speech alignments in 136 language pairs with a total of
418 thousand hours of speech. To evaluate the quality of this parallel speech,
we train bilingual speech-to-speech translation models on mined data only and
establish extensive baseline results on EuroParl-ST, VoxPopuli and FLEURS test
sets. Enabled by the multilinguality of SpeechMatrix, we also explore
multilingual speech-to-speech translation, a topic which was addressed by few
other works. We also demonstrate that model pre-training and sparse scaling
using Mixture-of-Experts bring large gains to translation performance. The
mined data and models are freely available.
