  Emotional Text-To-Speech (TTS) is an important task in the development of
systems (e.g., human-like dialogue agents) that require natural and emotional
speech. Existing approaches, however, only aim to produce emotional TTS for
seen speakers during training, without consideration of the generalization to
unseen speakers. In this paper, we propose ZET-Speech, a zero-shot adaptive
emotion-controllable TTS model that allows users to synthesize any speaker's
emotional speech using only a short, neutral speech segment and the target
emotion label. Specifically, to enable a zero-shot adaptive TTS model to
synthesize emotional speech, we propose domain adversarial learning and
guidance methods on the diffusion model. Experimental results demonstrate that
ZET-Speech successfully synthesizes natural and emotional speech with the
desired emotion for both seen and unseen speakers. Samples are at
https://ZET-Speech.github.io/ZET-Speech-Demo/.
