  One of the major shortcomings of variational autoencoders is the inability to
produce generations from the individual modalities of data originating from
mixture distributions. This is primarily due to the use of a simple isotropic
Gaussian as the prior for the latent code in the ancestral sampling procedure
for the data generations. We propose a novel formulation of variational
autoencoders, conditional prior VAE (CP-VAE), which learns to differentiate
between the individual mixture components and therefore allows for generations
from the distributional data clusters. We assume a two-level generative process
with a continuous (Gaussian) latent variable sampled conditionally on a
discrete (categorical) latent component. The new variational objective
naturally couples the learning of the posterior and prior conditionals, and the
learning of the latent categories encoding the multimodality of the original
data in an unsupervised manner. The data-dependent conditional priors are then
used to sample the continuous latent code when generating new samples from the
individual mixture components corresponding to the multimodal structure of the
original data. Our experimental results illustrate the generative performance
of our new model comparing to multiple baselines.
