  We introduce XTREME-S, a new benchmark to evaluate universal cross-lingual
speech representations in many languages. XTREME-S covers four task families:
speech recognition, classification, speech-to-text translation and retrieval.
Covering 102 languages from 10+ language families, 3 different domains and 4
task families, XTREME-S aims to simplify multilingual speech representation
evaluation, as well as catalyze research in "universal" speech representation
learning. This paper describes the new benchmark and establishes the first
speech-only and speech-text baselines using XLS-R and mSLAM on all downstream
tasks. We motivate the design choices and detail how to use the benchmark.
Datasets and fine-tuning scripts are made easily accessible at
https://hf.co/datasets/google/xtreme_s.
