  Variational inference often minimizes the "reverse" Kullbeck-Leibler (KL)
KL(q||p) from the approximate distribution q to the posterior p. Recent work
studies the "forward" KL KL(p||q), which unlike reverse KL does not lead to
variational approximations that underestimate uncertainty. This paper
introduces Transport Score Climbing (TSC), a method that optimizes KL(p||q) by
using Hamiltonian Monte Carlo (HMC) and a novel adaptive transport map. The
transport map improves the trajectory of HMC by acting as a change of variable
between the latent variable space and a warped space. TSC uses HMC samples to
dynamically train the transport map while optimizing KL(p||q). TSC leverages
synergies, where better transport maps lead to better HMC sampling, which then
leads to better transport maps. We demonstrate TSC on synthetic and real data.
We find that TSC achieves competitive performance when training variational
autoencoders on large-scale data.
