  In this paper, we propose a method for intermediating multiple speakers'
attributes and diversifying their voice characteristics in ``speaker
generation,'' an emerging task that aims to synthesize a nonexistent speaker's
naturally sounding voice. The conventional TacoSpawn-based speaker generation
method represents the distributions of speaker embeddings by Gaussian mixture
models (GMMs) conditioned with speaker attributes. Although this method enables
the sampling of various speakers from the speaker-attribute-aware GMMs, it is
not yet clear whether the learned distributions can represent speakers with an
intermediate attribute (i.e., mid-attribute). To this end, we propose an
optimal-transport-based method that interpolates the learned GMMs to generate
nonexistent speakers with mid-attribute (e.g., gender-neutral) voices. We
empirically validate our method and evaluate the naturalness of synthetic
speech and the controllability of two speaker attributes: gender and language
fluency. The evaluation results show that our method can control the generated
speakers' attributes by a continuous scalar value without statistically
significant degradation of speech naturalness.
