  Recently, more and more zero-shot voice conversion algorithms have been
proposed. As a fundamental part of zero-shot voice conversion, speaker
embeddings are the key to improving the converted speech's speaker similarity.
In this paper, we study the impact of speaker embeddings on zero-shot voice
conversion performance. To better represent the characteristics of the target
speaker and improve the speaker similarity in zero-shot voice conversion, we
propose a novel speaker representation method in this paper. Our method
combines the advantages of D-vector, global style token (GST) based speaker
representation and auxiliary supervision. Objective and subjective evaluations
show that the proposed method achieves a decent performance on zero-shot voice
conversion and significantly improves speaker similarity over D-vector and
GST-based speaker embedding.
