  Although text-to-speech (TTS) systems have significantly improved, most TTS
systems still have limitations in synthesizing speech with appropriate
phrasing. For natural speech synthesis, it is important to synthesize the
speech with a phrasing structure that groups words into phrases based on
semantic information. In this paper, we propose PuaseSpeech, a speech synthesis
system with a pre-trained language model and pause-based prosody modeling.
First, we introduce a phrasing structure encoder that utilizes a context
representation from the pre-trained language model. In the phrasing structure
encoder, we extract a speaker-dependent syntactic representation from the
context representation and then predict a pause sequence that separates the
input text into phrases. Furthermore, we introduce a pause-based word encoder
to model word-level prosody based on pause sequence. Experimental results show
PauseSpeech outperforms previous models in terms of naturalness. Furthermore,
in terms of objective evaluations, we can observe that our proposed methods
help the model decrease the distance between ground-truth and synthesized
speech. Audio samples are available at
https://jisang93.github.io/pausespeech-demo/.
