  The recently-developed WaveNet architecture is the current state of the art
in realistic speech synthesis, consistently rated as more natural sounding for
many different languages than any previous system. However, because WaveNet
relies on sequential generation of one audio sample at a time, it is poorly
suited to today's massively parallel computers, and therefore hard to deploy in
a real-time production setting. This paper introduces Probability Density
Distillation, a new method for training a parallel feed-forward network from a
trained WaveNet with no significant difference in quality. The resulting system
is capable of generating high-fidelity speech samples at more than 20 times
faster than real-time, and is deployed online by Google Assistant, including
serving multiple English and Japanese voices.
