  This paper investigates the in-context learning abilities of the Whisper
automatic speech recognition (ASR) models released by OpenAI. A novel
speech-based in-context learning (SICL) approach is proposed for test-time
adaptation, which can reduce the word error rates (WERs) with only a small
number of labelled speech samples without gradient descent. Language-level
adaptation experiments using Chinese dialects showed that when applying SICL to
isolated word ASR, consistent and considerable relative WER reductions can be
achieved using Whisper models of any size on two dialects, which is on average
32.3%. A k-nearest-neighbours-based in-context example selection technique can
be applied to further improve the efficiency of SICL, which can increase the
average relative WER reduction to 36.4%. The findings are verified using
speaker adaptation or continuous speech recognition tasks, and both achieved
considerable relative WER reductions. Detailed quantitative analyses are also
provided to shed light on SICL's adaptability to phonological variances and
dialect-specific lexical nuances.
