  Markov chain Monte Carlo (MCMC), such as Langevin dynamics, is valid for
approximating intractable distributions. However, its usage is limited in the
context of deep latent variable models owing to costly datapoint-wise sampling
iterations and slow convergence. This paper proposes the amortized Langevin
dynamics (ALD), wherein datapoint-wise MCMC iterations are entirely replaced
with updates of an encoder that maps observations into latent variables. This
amortization enables efficient posterior sampling without datapoint-wise
iterations. Despite its efficiency, we prove that ALD is valid as an MCMC
algorithm, whose Markov chain has the target posterior as a stationary
distribution under mild assumptions. Based on the ALD, we also present a new
deep latent variable model named the Langevin autoencoder (LAE). Interestingly,
the LAE can be implemented by slightly modifying the traditional autoencoder.
Using multiple synthetic datasets, we first validate that ALD can properly
obtain samples from target posteriors. We also evaluate the LAE on the image
generation task, and show that our LAE can outperform existing methods based on
variational inference, such as the variational autoencoder, and other
MCMC-based methods in terms of the test likelihood.
