  Deep generative models provide a systematic way to learn nonlinear data
distributions, through a set of latent variables and a nonlinear "generator"
function that maps latent points into the input space. The nonlinearity of the
generator imply that the latent space gives a distorted view of the input
space. Under mild conditions, we show that this distortion can be characterized
by a stochastic Riemannian metric, and demonstrate that distances and
interpolants are significantly improved under this metric. This in turn
improves probability distributions, sampling algorithms and clustering in the
latent space. Our geometric analysis further reveals that current generators
provide poor variance estimates and we propose a new generator architecture
with vastly improved variance estimates. Results are demonstrated on
convolutional and fully connected variational autoencoders, but the formalism
easily generalize to other deep generative models.
