  This paper presents a generative approach to speech enhancement based on a
recurrent variational autoencoder (RVAE). The deep generative speech model is
trained using clean speech signals only, and it is combined with a nonnegative
matrix factorization noise model for speech enhancement. We propose a
variational expectation-maximization algorithm where the encoder of the RVAE is
fine-tuned at test time, to approximate the distribution of the latent
variables given the noisy speech observations. Compared with previous
approaches based on feed-forward fully-connected architectures, the proposed
recurrent deep generative speech model induces a posterior temporal dynamic
over the latent variables, which is shown to improve the speech enhancement
results.
