  Although with progress in introducing auxiliary amortized inference models,
learning discrete latent variable models is still challenging. In this paper,
we show that the annoying difficulty of obtaining reliable stochastic gradients
for the inference model and the drawback of indirectly optimizing the target
log-likelihood can be gracefully addressed in a new method based on stochastic
approximation (SA) theory of the Robbins-Monro type. Specifically, we propose
to directly maximize the target log-likelihood and simultaneously minimize the
inclusive divergence between the posterior and the inference model. The
resulting learning algorithm is called joint SA (JSA). To the best of our
knowledge, JSA represents the first method that couples an SA version of the EM
(expectation-maximization) algorithm (SAEM) with an adaptive MCMC procedure.
Experiments on several benchmark generative modeling and structured prediction
tasks show that JSA consistently outperforms recent competitive algorithms,
with faster convergence, better final likelihoods, and lower variance of
gradient estimates.
