  We extend PAC-Bayesian theory to generative models and develop generalization
bounds for models based on the Wasserstein distance and the total variation
distance. Our first result on the Wasserstein distance assumes the instance
space is bounded, while our second result takes advantage of dimensionality
reduction. Our results naturally apply to Wasserstein GANs and Energy-Based
GANs, and our bounds provide new training objectives for these two. Although
our work is mainly theoretical, we perform numerical experiments showing
non-vacuous generalization bounds for Wasserstein GANs on synthetic datasets.
