  In the existing cross-speaker style transfer task, a source speaker with
multi-style recordings is necessary to provide the style for a target speaker.
However, it is hard for one speaker to express all expected styles. In this
paper, a more general task, which is to produce expressive speech by combining
any styles and timbres from a multi-speaker corpus in which each speaker has a
unique style, is proposed. To realize this task, a novel method is proposed.
This method is a Tacotron2-based framework but with a fine-grained text-based
prosody predicting module and a speaker identity controller. Experiments
demonstrate that the proposed method can successfully express a style of one
speaker with the timber of another speaker bypassing the dependency on a single
speaker's multi-style corpus. Moreover, the explicit prosody features used in
the prosody predicting module can increase the diversity of synthetic speech by
adjusting the value of prosody features.
