  We propose a learning-based filter that allows us to directly modify a
synthetic speech waveform into a natural speech waveform. Speech-processing
systems using a vocoder framework such as statistical parametric speech
synthesis and voice conversion are convenient especially for a limited number
of data because it is possible to represent and process interpretable acoustic
features over a compact space, such as the fundamental frequency (F0) and
mel-cepstrum. However, a well-known problem that leads to the quality
degradation of generated speech is an over-smoothing effect that eliminates
some detailed structure of generated/converted acoustic features. To address
this issue, we propose a synthetic-to-natural speech waveform conversion
technique that uses cycle-consistent adversarial networks and which does not
require any explicit assumption about speech waveform in adversarial learning.
In contrast to current techniques, since our modification is performed at the
waveform level, we expect that the proposed method will also make it possible
to generate `vocoder-less' sounding speech even if the input speech is
synthesized using a vocoder framework. The experimental results demonstrate
that our proposed method can 1) alleviate the over-smoothing effect of the
acoustic features despite the direct modification method used for the waveform
and 2) greatly improve the naturalness of the generated speech sounds.
