  An essential design decision for multilingual Neural Text-To-Speech (NTTS)
systems is how to represent input linguistic features within the model. Looking
at the wide variety of approaches in the literature, two main paradigms emerge,
unified and separate representations. The former uses a shared set of phonetic
tokens across languages, whereas the latter uses unique phonetic tokens for
each language. In this paper, we conduct a comprehensive study comparing
multilingual NTTS systems models trained with both representations. Our results
reveal that the unified approach consistently achieves better cross-lingual
synthesis with respect to both naturalness and accent. Separate representations
tend to have an order of magnitude more tokens than unified ones, which may
affect model capacity. For this reason, we carry out an ablation study to
understand the interaction of the representation type with the size of the
token embedding. We find that the difference between the two paradigms only
emerges above a certain threshold embedding size. This study provides strong
evidence that unified representations should be the preferred paradigm when
building multilingual NTTS systems.
