  Although numerous recent studies have suggested new frameworks for zero-shot
TTS using large-scale, real-world data, studies that focus on the
intelligibility of zero-shot TTS are relatively scarce. Zero-shot TTS demands
additional efforts to ensure clear pronunciation and speech quality due to its
inherent requirement of replacing a core parameter (speaker embedding or
acoustic prompt) with a new one at the inference stage. In this study, we
propose a zero-shot TTS model focused on intelligibility, which we refer to as
Intelli-Z. Intelli-Z learns speaker embeddings by using multi-speaker TTS as
its teacher and is trained with a cycle-consistency loss to include mismatched
text-speech pairs for training. Additionally, it selectively aggregates speaker
embeddings along the temporal dimension to minimize the interference of the
text content of reference speech at the inference stage. We substantiate the
effectiveness of the proposed methods with an ablation study. The Mean Opinion
Score (MOS) increases by 9% for unseen speakers when the first two methods are
applied, and it further improves by 16% when selective temporal aggregation is
applied.
