  This paper proposes a speech synthesis system that allows users to specify
and control the acoustic characteristics of a speaker by means of prompts
describing the speaker's traits of synthesized speech. Unlike previous
approaches, our method utilizes listener impressions to construct prompts,
which are easier to collect and align more naturally with everyday descriptions
of speaker traits. We adopt the Low-rank Adaptation (LoRA) technique to swiftly
tailor a pre-trained language model to our needs, facilitating the extraction
of speaker-related traits from the prompt text. Besides, different from other
prompt-driven text-to-speech (TTS) systems, we separate the prompt-to-speaker
module from the multi-speaker TTS system, enhancing system flexibility and
compatibility with various pre-trained multi-speaker TTS systems. Moreover, for
the prompt-to-speaker characteristic module, we also compared the
discriminative method and flow-matching based generative method and we found
that combining both methods can help the system simultaneously capture
speaker-related information from prompts better and generate speech with higher
fidelity.
