  A learned generative model often produces biased statistics relative to the
underlying data distribution. A standard technique to correct this bias is
importance sampling, where samples from the model are weighted by the
likelihood ratio under model and true distributions. When the likelihood ratio
is unknown, it can be estimated by training a probabilistic classifier to
distinguish samples from the two distributions. We employ this likelihood-free
importance weighting method to correct for the bias in generative models. We
find that this technique consistently improves standard goodness-of-fit metrics
for evaluating the sample quality of state-of-the-art deep generative models,
suggesting reduced bias. Finally, we demonstrate its utility on representative
applications in a) data augmentation for classification using generative
adversarial networks, and b) model-based policy evaluation using off-policy
data.
