  Scaling text-to-speech (TTS) with autoregressive language model (LM) to
large-scale datasets by quantizing waveform into discrete speech tokens is
making great progress to capture the diversity and expressiveness in human
speech, but the speech reconstruction quality from discrete speech token is far
from satisfaction depending on the compressed speech token compression ratio.
Generative diffusion models trained with score-matching loss and continuous
normalized flow trained with flow-matching loss have become prominent in
generation of images as well as speech. LM based TTS systems usually quantize
speech into discrete tokens and generate these tokens autoregressively, and
finally use a diffusion model to up sample coarse-grained speech tokens into
fine-grained codec features or mel-spectrograms before reconstructing into
waveforms with vocoder, which has a high latency and is not realistic for real
time speech applications. In this paper, we systematically investigate varied
diffusion models for up sampling stage, which is the main bottleneck for
streaming synthesis of LM and diffusion-based architecture, we present the
model architecture, objective and subjective metrics to show quality and
efficiency improvement.
