  Recent speech synthesis systems based on sampling from autoregressive neural
networks models can generate speech almost undistinguishable from human
recordings. However, these models require large amounts of data. This paper
shows that the lack of data from one speaker can be compensated with data from
other speakers. The naturalness of Tacotron2-like models trained on a blend of
5k utterances from 7 speakers is better than that of speaker dependent models
trained on 15k utterances, but in terms of stability multi-speaker models are
always more stable. We also demonstrate that models mixing only 1250 utterances
from a target speaker with 5k utterances from another 6 speakers can produce
significantly better quality than state-of-the-art DNN-guided unit selection
systems trained on more than 10 times the data from the target speaker.
