  This paper proposes a method for generating speech from filterbank mel
frequency cepstral coefficients (MFCC), which are widely used in speech
applications, such as ASR, but are generally considered unusable for speech
synthesis. First, we predict fundamental frequency and voicing information from
MFCCs with an autoregressive recurrent neural net. Second, the spectral
envelope information contained in MFCCs is converted to all-pole filters, and a
pitch-synchronous excitation model matched to these filters is trained.
Finally, we introduce a generative adversarial network -based noise model to
add a realistic high-frequency stochastic component to the modeled excitation
signal. The results show that high quality speech reconstruction can be
obtained, given only MFCC information at test time.
