  Prevalent ungrammatical expressions and disfluencies in spontaneous speech
from second language (L2) learners pose unique challenges to Automatic Speech
Recognition (ASR) systems. However, few datasets are tailored to L2 learner
speech. We publicly release LearnerVoice, a dataset consisting of 50.04 hours
of audio and transcriptions of L2 learners' spontaneous speech. Our linguistic
analysis reveals that transcriptions in our dataset contain L2S (L2 learner's
Spontaneous speech) features, consisting of ungrammatical expressions and
disfluencies (e.g., filler words, word repetitions, self-repairs, false
starts), significantly more than native speech datasets. Fine-tuning
whisper-small.en with LearnerVoice achieves a WER of 10.26%, 44.2% lower than
vanilla whisper-small.en. Furthermore, our qualitative analysis indicates that
54.2% of errors from the vanilla model on LearnerVoice are attributable to L2S
features, with 48.1% of them being reduced in the fine-tuned model.
