  Children's speech recognition is a vital, yet largely overlooked domain when
building inclusive speech technologies. The major challenge impeding progress
in this domain is the lack of adequate child speech corpora; however, recent
advances in self-supervised learning have created a new opportunity for
overcoming this problem of data scarcity. In this paper, we leverage
self-supervised adult speech representations and use three well-known child
speech corpora to build models for children's speech recognition. We assess the
performance of fine-tuning on both native and non-native children's speech,
examine the effect of cross-domain child corpora, and investigate the minimum
amount of child speech required to fine-tune a model which outperforms a
state-of-the-art adult model. We also analyze speech recognition performance
across children's ages. Our results demonstrate that fine-tuning with
cross-domain child corpora leads to relative improvements of up to 46.08% and
45.53% for native and non-native child speech respectively, and absolute
improvements of 14.70% and 31.10%. We also show that with as little as 5 hours
of transcribed children's speech, it is possible to fine-tune a children's
speech recognition system that outperforms a state-of-the-art adult model
fine-tuned on 960 hours of adult speech.
