  Accents play a pivotal role in shaping human communication, enhancing our
ability to convey and comprehend messages with clarity and cultural nuance.
While there has been significant progress in Automatic Speech Recognition
(ASR), African-accented English ASR has been understudied due to a lack of
training datasets, which are often expensive to create and demand colossal
human labor. Combining several active learning paradigms and the core-set
approach, we propose a new multi-rounds adaptation process that uses epistemic
uncertainty to automate the annotation process, significantly reducing the
associated costs and human labor. This novel method streamlines data annotation
and strategically selects data samples contributing most to model uncertainty,
enhancing training efficiency. We define a new U-WER metric to track model
adaptation to hard accents. We evaluate our approach across several domains,
datasets, and high-performing speech models. Our results show that our approach
leads to a 27\% WER relative average improvement while requiring on average
45\% less data than established baselines. Our approach also improves
out-of-distribution generalization for very low-resource accents, demonstrating
its viability for building generalizable ASR models in the context of accented
African ASR. We open-source the code here:
https://github.com/bonaventuredossou/active_learning_african_asr.
