  In this paper, a neural network named Sequence-to-sequence ConvErsion NeTwork
(SCENT) is presented for acoustic modeling in voice conversion. At training
stage, a SCENT model is estimated by aligning the feature sequences of source
and target speakers implicitly using attention mechanism. At conversion stage,
acoustic features and durations of source utterances are converted
simultaneously using the unified acoustic model. Mel-scale spectrograms are
adopted as acoustic features which contain both excitation and vocal tract
descriptions of speech signals. The bottleneck features extracted from source
speech using an automatic speech recognition (ASR) model are appended as
auxiliary input. A WaveNet vocoder conditioned on Mel-spectrograms is built to
reconstruct waveforms from the outputs of the SCENT model. It is worth noting
that our proposed method can achieve appropriate duration conversion which is
difficult in conventional methods. Experimental results show that our proposed
method obtained better objective and subjective performance than the baseline
methods using Gaussian mixture models (GMM) and deep neural networks (DNN) as
acoustic models. This proposed method also outperformed our previous work which
achieved the top rank in Voice Conversion Challenge 2018. Ablation tests
further confirmed the effectiveness of several components in our proposed
method.
