  Deep generative models are becoming widely used across science and industry
for a variety of purposes. A common challenge is achieving a precise implicit
or explicit representation of the data probability density. Recent proposals
have suggested using classifier weights to refine the learned density of deep
generative models. We extend this idea to all types of generative models and
show how latent space refinement via iterated generative modeling can
circumvent topological obstructions and improve precision. This methodology
also applies to cases were the target model is non-differentiable and has many
internal latent dimensions which must be marginalized over before refinement.
We demonstrate our Latent Space Refinement (LaSeR) protocol on a variety of
examples, focusing on the combinations of Normalizing Flows and Generative
Adversarial Networks.
