  Recent research using pre-trained transformer models suggests that just 10
minutes of transcribed speech may be enough to fine-tune such a model for
automatic speech recognition (ASR) -- at least if we can also leverage vast
amounts of text data (803 million tokens). But is that much text data
necessary? We study the use of different amounts of text data, both for
creating a lexicon that constrains ASR decoding to possible words (e.g. *dogz
vs. dogs), and for training larger language models that bias the system toward
probable word sequences (e.g. too dogs vs. two dogs). We perform experiments
using 10 minutes of transcribed speech from English (for replicating prior
work) and two additional pairs of languages differing in the availability of
supplemental text data: Gronings and Frisian (~7.5M token corpora available),
and Besemah and Nasal (only small lexica available). For all languages, we
found that using only a lexicon did not appreciably improve ASR performance.
For Gronings and Frisian, we found that lexica and language models derived from
'novel-length' 80k token subcorpora reduced the word error rate (WER) to 39% on
average. Our findings suggest that where a text corpus in the upper tens of
thousands of tokens or more is available, fine-tuning a transformer model with
just tens of minutes of transcribed speech holds some promise towards obtaining
human-correctable transcriptions near the 30% WER rule-of-thumb.
