  Learning latent variable models with stochastic variational inference is
challenging when the approximate posterior is far from the true posterior, due
to high variance in the gradient estimates. We propose a novel rejection
sampling step that discards samples from the variational posterior which are
assigned low likelihoods by the model. Our approach provides an arbitrarily
accurate approximation of the true posterior at the expense of extra
computation. Using a new gradient estimator for the resulting unnormalized
proposal distribution, we achieve average improvements of 3.71 nats and 0.21
nats over state-of-the-art single-sample and multi-sample alternatives
respectively for estimating marginal log-likelihoods using sigmoid belief
networks on the MNIST dataset.
