  Markov Chain Monte Carlo (MCMC) methods sample from unnormalized probability
distributions and offer guarantees of exact sampling. However, in the
continuous case, unfavorable geometry of the target distribution can greatly
limit the efficiency of MCMC methods. Augmenting samplers with neural networks
can potentially improve their efficiency. Previous neural network based
samplers were trained with objectives that either did not explicitly encourage
exploration, or used a L2 jump objective which could only be applied to well
structured distributions. Thus it seems promising to instead maximize the
proposal entropy for adapting the proposal to distributions of any shape. To
allow direct optimization of the proposal entropy, we propose a neural network
MCMC sampler that has a flexible and tractable proposal distribution.
Specifically, our network architecture utilizes the gradient of the target
distribution for generating proposals. Our model achieves significantly higher
efficiency than previous neural network MCMC techniques in a variety of
sampling tasks. Further, the sampler is applied on training of a convergent
energy-based model of natural images. The adaptive sampler achieves unbiased
sampling with significantly higher proposal entropy than Langevin dynamics
sampler.
