  Speech foundation models (SFMs) have achieved state-of-the-art results for
various speech tasks in supervised (e.g. Whisper) or self-supervised systems
(e.g. WavLM). However, the performance of SFMs for child ASR has not been
systematically studied. In addition, there is no benchmark for child ASR with
standard evaluations, making the comparisons of novel ideas difficult. In this
paper, we initiate and present a comprehensive benchmark on several child
speech databases based on various SFMs (Whisper, Wav2vec2.0, HuBERT, and
WavLM). Moreover, we investigate finetuning strategies by comparing various
data augmentation and parameter-efficient finetuning (PEFT) methods. We observe
that the behaviors of these methods are different when the model size
increases. For example, PEFT matches the performance of full finetuning for
large models but worse for small models. To stabilize finetuning using
augmented data, we propose a perturbation invariant finetuning (PIF) loss as a
regularization.
