  Despite recent availability of large transcribed Kinyarwanda speech data,
achieving robust speech recognition for Kinyarwanda is still challenging. In
this work, we show that using self-supervised pre-training, following a simple
curriculum schedule during fine-tuning and using semi-supervised learning to
leverage large unlabelled speech data significantly improve speech recognition
performance for Kinyarwanda. Our approach focuses on using public domain data
only. A new studio-quality speech dataset is collected from a public website,
then used to train a clean baseline model. The clean baseline model is then
used to rank examples from a more diverse and noisy public dataset, defining a
simple curriculum training schedule. Finally, we apply semi-supervised learning
to label and learn from large unlabelled data in five successive generations.
Our final model achieves 3.2% word error rate (WER) on the new dataset and
15.6% WER on Mozilla Common Voice benchmark, which is state-of-the-art to the
best of our knowledge. Our experiments also indicate that using syllabic rather
than character-based tokenization results in better speech recognition
performance for Kinyarwanda.
