  Variational autoencoders (VAEs) and generative adversarial networks (GANs)
enjoy an intuitive connection to manifold learning: in training the
decoder/generator is optimized to approximate a homeomorphism between the data
distribution and the sampling space. This is a construction that strives to
define the data manifold. A major obstacle to VAEs and GANs, however, is
choosing a suitable prior that matches the data topology. Well-known
consequences of poorly picked priors are posterior and mode collapse. To our
knowledge, no existing method sidesteps this user choice. Conversely,
$\textit{diffusion maps}$ automatically infer the data topology and enjoy a
rigorous connection to manifold learning, but do not scale easily or provide
the inverse homeomorphism (i.e. decoder/generator). We propose a method that
combines these approaches into a generative model that inherits the asymptotic
guarantees of $\textit{diffusion maps}$ while preserving the scalability of
deep models. We prove approximation theoretic results for the dimension
dependence of our proposed method. Finally, we demonstrate the effectiveness of
our method with various real and synthetic datasets.
