  We investigated the training of a shared model for both text-to-speech (TTS)
and voice conversion (VC) tasks. We propose using an extended model
architecture of Tacotron, that is a multi-source sequence-to-sequence model
with a dual attention mechanism as the shared model for both the TTS and VC
tasks. This model can accomplish these two different tasks respectively
according to the type of input. An end-to-end speech synthesis task is
conducted when the model is given text as the input while a
sequence-to-sequence voice conversion task is conducted when it is given the
speech of a source speaker as the input. Waveform signals are generated by
using WaveNet, which is conditioned by using a predicted mel-spectrogram. We
propose jointly training a shared model as a decoder for a target speaker that
supports multiple sources. Listening experiments show that our proposed
multi-source encoder-decoder model can efficiently achieve both the TTS and VC
tasks.
