  While modern TTS technologies have made significant advancements in audio
quality, there is still a lack of behavior naturalness compared to conversing
with people. We propose a style-embedded TTS system that generates styled
responses based on the speech query style. To achieve this, the system includes
a style extraction model that extracts a style embedding from the speech query,
which is then used by the TTS to produce a matching response. We faced two main
challenges: 1) only a small portion of the TTS training dataset has style
labels, which is needed to train a multi-style TTS that respects different
style embeddings during inference. 2) The TTS system and the style extraction
model have disjoint training datasets. We need consistent style labels across
these two datasets so that the TTS can learn to respect the labels produced by
the style extraction model during inference. To solve these, we adopted a
semi-supervised approach that uses the style extraction model to create style
labels for the TTS dataset and applied transfer learning to learn the style
embedding jointly. Our experiment results show user preference for the styled
TTS responses and demonstrate the style-embedded TTS system's capability of
mimicking the speech query style.
