  Existing Markov Chain Monte Carlo (MCMC) methods are either based on
general-purpose and domain-agnostic schemes which can lead to slow convergence,
or hand-crafting of problem-specific proposals by an expert. We propose
A-NICE-MC, a novel method to train flexible parametric Markov chain kernels to
produce samples with desired properties. First, we propose an efficient
likelihood-free adversarial training method to train a Markov chain and mimic a
given data distribution. Then, we leverage flexible volume preserving flows to
obtain parametric kernels for MCMC. Using a bootstrap approach, we show how to
train efficient Markov chains to sample from a prescribed posterior
distribution by iteratively improving the quality of both the model and the
samples. A-NICE-MC provides the first framework to automatically design
efficient domain-specific MCMC proposals. Empirical results demonstrate that
A-NICE-MC combines the strong guarantees of MCMC with the expressiveness of
deep neural networks, and is able to significantly outperform competing methods
such as Hamiltonian Monte Carlo.
