  Voice conversion has made great progress in the past few years under the
studio-quality test scenario in terms of speech quality and speaker similarity.
However, in real applications, test speech from source speaker or target
speaker can be corrupted by various environment noises, which seriously degrade
the speech quality and speaker similarity. In this paper, we propose a novel
encoder-decoder based noise-robust voice conversion framework, which consists
of a speaker encoder, a content encoder, a decoder, and two domain adversarial
neural networks. Specifically, we integrate disentangling speaker and content
representation technique with domain adversarial training technique. Domain
adversarial training makes speaker representations and content representations
extracted by speaker encoder and content encoder from clean speech and noisy
speech in the same space, respectively. In this way, the learned speaker and
content representations are noise-invariant. Therefore, the two noise-invariant
representations can be taken as input by the decoder to predict the clean
converted spectrum. The experimental results demonstrate that our proposed
method can synthesize clean converted speech under noisy test scenarios, where
the source speech and target speech can be corrupted by seen or unseen noise
types during the training process. Additionally, both speech quality and
speaker similarity are improved.
