  This paper explains our work in developing new acoustic models for automated
speech recognition (ASR) at KBLab, the infrastructure for data-driven research
at the National Library of Sweden (KB). We evaluate different approaches for a
viable speech-to-text pipeline for audiovisual resources in Swedish, using the
wav2vec 2.0 architecture in combination with speech corpuses created from KB's
collections. These approaches include pretraining an acoustic model for Swedish
from the ground up, and fine-tuning existing monolingual and multilingual
models. The collections-based corpuses we use have been sampled from millions
of hours of speech, with a conscious attempt to balance regional dialects to
produce a more representative, and thus more democratic, model. The acoustic
model this enabled, "VoxRex", outperforms existing models for Swedish ASR. We
also evaluate combining this model with various pretrained language models,
which further enhanced performance. We conclude by highlighting the potential
of such technology for cultural heritage institutions with vast collections of
previously unlabelled audiovisual data. Our models are released for further
exploration and research here: https://huggingface.co/KBLab.
