  In this work, we present DiffVoice, a novel text-to-speech model based on
latent diffusion. We propose to first encode speech signals into a phoneme-rate
latent representation with a variational autoencoder enhanced by adversarial
training, and then jointly model the duration and the latent representation
with a diffusion model. Subjective evaluations on LJSpeech and LibriTTS
datasets demonstrate that our method beats the best publicly available systems
in naturalness. By adopting recent generative inverse problem solving
algorithms for diffusion models, DiffVoice achieves the state-of-the-art
performance in text-based speech editing, and zero-shot adaptation.
