  As edge-based automatic speech recognition (ASR) technologies become
increasingly prevalent for the development of intelligent and personalized
assistants, three important challenges must be addressed for these
resource-constrained ASR models, i.e., adaptivity, incrementality, and
inclusivity. We propose a novel ASR framework, PI-Whisper, in this work and
show how it can improve an ASR's recognition capabilities adaptively by
identifying different speakers' characteristics in real-time, how such an
adaption can be performed incrementally without repetitive retraining, and how
it can improve the equity and fairness for diverse speaker groups. More
impressively, our proposed PI-Whisper framework attains all of these nice
properties while still achieving state-of-the-art accuracy with up to 13.7%
reduction of the word error rate (WER) with linear scalability with respect to
computing resources.
