  We present doubly stochastic gradient MCMC, a simple and generic method for
(approximate) Bayesian inference of deep generative models (DGMs) in a
collapsed continuous parameter space. At each MCMC sampling step, the algorithm
randomly draws a mini-batch of data samples to estimate the gradient of
log-posterior and further estimates the intractable expectation over hidden
variables via a neural adaptive importance sampler, where the proposal
distribution is parameterized by a deep neural network and learnt jointly. We
demonstrate the effectiveness on learning various DGMs in a wide range of
tasks, including density estimation, data generation and missing data
imputation. Our method outperforms many state-of-the-art competitors.
