  Variational Autoencoder (VAE), a simple and effective deep generative model,
has led to a number of impressive empirical successes and spawned many advanced
variants and theoretical investigations. However, recent studies demonstrate
that, when equipped with expressive generative distributions (aka. decoders),
VAE suffers from learning uninformative latent representations with the
observation called KL Varnishing, in which case VAE collapses into an
unconditional generative model. In this work, we introduce mutual
posterior-divergence regularization, a novel regularization that is able to
control the geometry of the latent space to accomplish meaningful
representation learning, while achieving comparable or superior capability of
density estimation. Experiments on three image benchmark datasets demonstrate
that, when equipped with powerful decoders, our model performs well both on
density estimation and representation learning.
