  We propose a new family of optimization criteria for variational
auto-encoding models, generalizing the standard evidence lower bound. We
provide conditions under which they recover the data distribution and learn
latent features, and formally show that common issues such as blurry samples
and uninformative latent features arise when these conditions are not met.
Based on these new insights, we propose a new sequential VAE model that can
generate sharp samples on the LSUN image dataset based on pixel-wise
reconstruction loss, and propose an optimization criterion that encourages
unsupervised learning of informative latent features.
