  Speaker verification systems often degrade significantly when there is a
language mismatch between training and testing data. Being able to improve
cross-lingual speaker verification system using unlabeled data can greatly
increase the robustness of the system and reduce human labeling costs. In this
study, we introduce an unsupervised Adversarial Discriminative Domain
Adaptation (ADDA) method to effectively learn an asymmetric mapping that adapts
the target domain encoder to the source domain, where the target domain and
source domain are speech data from different languages. ADDA, together with a
popular Domain Adversarial Training (DAT) approach, are evaluated on a
cross-lingual speaker verification task: the training data is in English from
NIST SRE04-08, Mixer 6 and Switchboard, and the test data is in Chinese from
AISHELL-I. We show that with the ADDA adaptation, Equal Error Rate (EER) of the
x-vector system decreases from 9.331\% to 7.645\%, relatively 18.07\% reduction
of EER, and 6.32\% reduction from DAT as well. Further data analysis of ADDA
adapted speaker embedding shows that the learned speaker embeddings can perform
well on speaker classification for the target domain data, and are less
dependent with respect to the shift in language.
