  With 4.5 million hours of English speech from 10 different sources across 120
countries and models of up to 10 billion parameters, we explore the frontiers
of scale for automatic speech recognition. We propose data selection techniques
to efficiently scale training data to find the most valuable samples in massive
datasets. To efficiently scale model sizes, we leverage various optimizations
such as sparse transducer loss and model sharding. By training 1-10B parameter
universal English ASR models, we push the limits of speech recognition
performance across many domains. Furthermore, our models learn powerful speech
representations with zero and few-shot capabilities on novel domains and styles
of speech, exceeding previous results across multiple in-house and public
benchmarks. For speakers with disorders due to brain damage, our best zero-shot
and few-shot models achieve 22% and 60% relative improvement on the AphasiaBank
test set, respectively, while realizing the best performance on public social
media videos. Furthermore, the same universal model reaches equivalent
performance with 500x less in-domain data on the SPGISpeech financial-domain
dataset.
