  We propose a family of novel hierarchical Bayesian deep auto-encoder models
capable of identifying disentangled factors of variability in data. While many
recent attempts at factor disentanglement have focused on sophisticated
learning objectives within the VAE framework, their choice of a standard normal
as the latent factor prior is both suboptimal and detrimental to performance.
Our key observation is that the disentangled latent variables responsible for
major sources of variability, the relevant factors, can be more appropriately
modeled using long-tail distributions. The typical Gaussian priors are, on the
other hand, better suited for modeling of nuisance factors. Motivated by this,
we extend the VAE to a hierarchical Bayesian model by introducing hyper-priors
on the variances of Gaussian latent priors, mimicking an infinite mixture,
while maintaining tractable learning and inference of the traditional VAEs.
This analysis signifies the importance of partitioning and treating in a
different manner the latent dimensions corresponding to relevant factors and
nuisances. Our proposed models, dubbed Bayes-Factor-VAEs, are shown to
outperform existing methods both quantitatively and qualitatively in terms of
latent disentanglement across several challenging benchmark tasks.
