  This paper proposes a human-in-the-loop speaker-adaptation method for
multi-speaker text-to-speech. With a conventional speaker-adaptation method, a
target speaker's embedding vector is extracted from his/her reference speech
using a speaker encoder trained on a speaker-discriminative task. However, this
method cannot obtain an embedding vector for the target speaker when the
reference speech is unavailable. Our method is based on a human-in-the-loop
optimization framework, which incorporates a user to explore the
speaker-embedding space to find the target speaker's embedding. The proposed
method uses a sequential line search algorithm that repeatedly asks a user to
select a point on a line segment in the embedding space. To efficiently choose
the best speech sample from multiple stimuli, we also developed a system in
which a user can switch between multiple speakers' voices for each phoneme
while looping an utterance. Experimental results indicate that the proposed
method can achieve comparable performance to the conventional one in objective
and subjective evaluations even if reference speech is not used as the input of
a speaker encoder directly.
