  Markov Chain Monte Carlo (MCMC) methods are a powerful tool for computation
with complex probability distributions. However the performance of such methods
is critically dependant on properly tuned parameters, most of which are
difficult if not impossible to know a priori for a given target distribution.
Adaptive MCMC methods aim to address this by allowing the parameters to be
updated during sampling based on previous samples from the chain at the expense
of requiring a new theoretical analysis to ensure convergence. In this work we
extend the convergence theory of adaptive MCMC methods to a new class of
methods built on a powerful class of parametric density estimators known as
normalizing flows. In particular, we consider an independent
Metropolis-Hastings sampler where the proposal distribution is represented by a
normalizing flow whose parameters are updated using stochastic gradient
descent. We explore the practical performance of this procedure on both
synthetic settings and in the analysis of a physical field system and compare
it against both adaptive and non-adaptive MCMC methods.
