  Cross-speaker style transfer in speech synthesis aims at transferring a style
from source speaker to synthesized speech of a target speaker's timbre. In most
previous methods, the synthesized fine-grained prosody features often represent
the source speaker's average style, similar to the one-to-many problem(i.e.,
multiple prosody variations correspond to the same text). In response to this
problem, a strength-controlled semi-supervised style extractor is proposed to
disentangle the style from content and timbre, improving the representation and
interpretability of the global style embedding, which can alleviate the
one-to-many mapping and data imbalance problems in prosody prediction. A
hierarchical prosody predictor is proposed to improve prosody modeling. We find
that better style transfer can be achieved by using the source speaker's
prosody features that are easily predicted. Additionally, a
speaker-transfer-wise cycle consistency loss is proposed to assist the model in
learning unseen style-timbre combinations during the training phase.
Experimental results show that the method outperforms the baseline. We provide
a website with audio samples.
