  We introduce a technique for augmenting neural text-to-speech (TTS) with
lowdimensional trainable speaker embeddings to generate different voices from a
single model. As a starting point, we show improvements over the two
state-ofthe-art approaches for single-speaker neural TTS: Deep Voice 1 and
Tacotron. We introduce Deep Voice 2, which is based on a similar pipeline with
Deep Voice 1, but constructed with higher performance building blocks and
demonstrates a significant audio quality improvement over Deep Voice 1. We
improve Tacotron by introducing a post-processing neural vocoder, and
demonstrate a significant audio quality improvement. We then demonstrate our
technique for multi-speaker speech synthesis for both Deep Voice 2 and Tacotron
on two multi-speaker TTS datasets. We show that a single neural TTS system can
learn hundreds of unique voices from less than half an hour of data per
speaker, while achieving high audio quality synthesis and preserving the
speaker identities almost perfectly.
