  In this paper, we study two aspects of the variational autoencoder (VAE): the
prior distribution over the latent variables and its corresponding posterior.
First, we decompose the learning of VAEs into layerwise density estimation, and
argue that having a flexible prior is beneficial to both sample generation and
inference. Second, we analyze the family of inverse autoregressive flows
(inverse AF) and show that with further improvement, inverse AF could be used
as universal approximation to any complicated posterior. Our analysis results
in a unified approach to parameterizing a VAE, without the need to restrict
ourselves to use factorial Gaussians in the latent real space.
