  End-to-end automatic speech recognition (ASR) can achieve promising
performance with large-scale training data. However, it is known that domain
mismatch between training and testing data often leads to a degradation of
recognition accuracy. In this work, we focus on the unsupervised domain
adaptation for ASR and propose CMatch, a Character-level distribution matching
method to perform fine-grained adaptation between each character in two
domains. First, to obtain labels for the features belonging to each character,
we achieve frame-level label assignment using the Connectionist Temporal
Classification (CTC) pseudo labels. Then, we match the character-level
distributions using Maximum Mean Discrepancy. We train our algorithm using the
self-training technique. Experiments on the Libri-Adapt dataset show that our
proposed approach achieves 14.39% and 16.50% relative Word Error Rate (WER)
reduction on both cross-device and cross-environment ASR. We also
comprehensively analyze the different strategies for frame-level label
assignment and Transformer adaptations.
