  We develop nested variational inference (NVI), a family of methods that learn
proposals for nested importance samplers by minimizing an forward or reverse KL
divergence at each level of nesting. NVI is applicable to many commonly-used
importance sampling strategies and provides a mechanism for learning
intermediate densities, which can serve as heuristics to guide the sampler. Our
experiments apply NVI to (a) sample from a multimodal distribution using a
learned annealing path (b) learn heuristics that approximate the likelihood of
future observations in a hidden Markov model and (c) to perform amortized
inference in hierarchical deep generative models. We observe that optimizing
nested objectives leads to improved sample quality in terms of log average
weight and effective sample size.
