  We propose denoising diffusion variational inference (DDVI), an approximate
inference algorithm for latent variable models which relies on diffusion models
as flexible variational posteriors. Specifically, our method introduces an
expressive class of approximate posteriors with auxiliary latent variables that
perform diffusion in latent space by reversing a user-specified noising
process. We fit these models by optimizing a lower bound on the marginal
likelihood inspired by the wake-sleep algorithm. Our method is easy to
implement (it fits a regularized extension of the ELBO), is compatible with
black-box variational inference, and outperforms alternative classes of
approximate posteriors based on normalizing flows or adversarial networks. It
increases the expressivity of flow-based methods via non-invertible deep
recurrent architectures and avoids the instability of adversarial methods. We
use DDVI on a motivating task in biology -- inferring latent ancestry from
human genomes -- and we find that it outperforms strong baselines on the
Thousand Genomes dataset.
