  Neural text-to-speech (TTS) has achieved human-like synthetic speech for
single-speaker, single-language synthesis. Multilingual TTS systems are limited
to resource-rich languages due to the lack of large paired text and
studio-quality audio data. In most cases, TTS systems are built using a single
speaker's voice. However, there is growing interest in developing systems that
can synthesize voices for new speakers using only a few seconds of their
speech. This paper presents ZMM-TTS, a multilingual and multispeaker framework
utilizing quantized latent speech representations from a large-scale,
pre-trained, self-supervised model. Our paper is the first to incorporate the
representations from text-based and speech-based self-supervised learning
models into multilingual speech synthesis tasks. We conducted comprehensive
subjective and objective evaluations through a series of experiments. Our model
has been proven effective in terms of speech naturalness and similarity for
both seen and unseen speakers in six high-resource languages. We also tested
the efficiency of our method on two hypothetical low-resource languages. The
results are promising, indicating that our proposed approach can synthesize
audio that is intelligible and has a high degree of similarity to the target
speaker's voice, even without any training data for the new, unseen language.
