  Zero-shot multi-speaker text-to-speech (ZSM-TTS) models aim to generate a
speech sample with the voice characteristic of an unseen speaker. The main
challenge of ZSM-TTS is to increase the overall speaker similarity for unseen
speakers. One of the most successful speaker conditioning methods for
flow-based multi-speaker text-to-speech (TTS) models is to utilize the
functions which predict the scale and bias parameters of the affine coupling
layers according to the given speaker embedding vector. In this letter, we
improve on the previous speaker conditioning method by introducing a
speaker-normalized affine coupling (SNAC) layer which allows for unseen speaker
speech synthesis in a zero-shot manner leveraging a normalization-based
conditioning technique. The newly designed coupling layer explicitly normalizes
the input by the parameters predicted from a speaker embedding vector while
training, enabling an inverse process of denormalizing for a new speaker
embedding at inference. The proposed conditioning scheme yields the
state-of-the-art performance in terms of the speech quality and speaker
similarity in a ZSM-TTS setting.
