  Cross-speaker style transfer is crucial to the applications of multi-style
and expressive speech synthesis at scale. It does not require the target
speakers to be experts in expressing all styles and to collect corresponding
recordings for model training. However, the performances of existing style
transfer methods are still far behind real application needs. The root causes
are mainly twofold. Firstly, the style embedding extracted from single
reference speech can hardly provide fine-grained and appropriate prosody
information for arbitrary text to synthesize. Secondly, in these models the
content/text, prosody, and speaker timbre are usually highly entangled, it's
therefore not realistic to expect a satisfied result when freely combining
these components, such as to transfer speaking style between speakers. In this
paper, we propose a cross-speaker style transfer text-to-speech (TTS) model
with explicit prosody bottleneck. The prosody bottleneck builds up the kernels
accounting for speaking style robustly, and disentangles the prosody from
content and speaker timbre, therefore guarantees high quality cross-speaker
style transfer. Evaluation result shows the proposed method even achieves
on-par performance with source speaker's speaker-dependent (SD) model in
objective measurement of prosody, and significantly outperforms the cycle
consistency and GMVAE-based baselines in objective and subjective evaluations.
