  Expressive speech synthesis, like audiobook synthesis, is still challenging
for style representation learning and prediction. Deriving from reference audio
or predicting style tags from text requires a huge amount of labeled data,
which is costly to acquire and difficult to define and annotate accurately. In
this paper, we propose a novel framework for learning style representation from
abundant plain text in a self-supervised manner. It leverages an emotion
lexicon and uses contrastive learning and deep clustering. We further integrate
the style representation as a conditioned embedding in a multi-style
Transformer TTS. Comparing with multi-style TTS by predicting style tags
trained on the same dataset but with human annotations, our method achieves
improved results according to subjective evaluations on both in-domain and
out-of-domain test sets in audiobook speech. Moreover, with implicit
context-aware style representation, the emotion transition of synthesized audio
in a long paragraph appears more natural. The audio samples are available on
the demo web.
