  In our previous work, we proposed a language-independent speaker
anonymization system based on self-supervised learning models. Although the
system can anonymize speech data of any language, the anonymization was
imperfect, and the speech content of the anonymized speech was distorted. This
limitation is more severe when the input speech is from a domain unseen in the
training data. This study analyzed the bottleneck of the anonymization system
under unseen conditions. It was found that the domain (e.g., language and
channel) mismatch between the training and test data affected the neural
waveform vocoder and anonymized speaker vectors, which limited the performance
of the whole system. Increasing the training data diversity for the vocoder was
found to be helpful to reduce its implicit language and channel dependency.
Furthermore, a simple correlation-alignment-based domain adaption strategy was
found to be significantly effective to alleviate the mismatch on the anonymized
speaker vectors. Audio samples and source code are available online.
