  We explain why directly changing the prior can be a surprisingly ineffective
mechanism for incorporating inductive biases into VAEs, and introduce a simple
and effective alternative approach: Intermediary Latent Space VAEs(InteL-VAEs).
InteL-VAEs use an intermediary set of latent variables to control the
stochasticity of the encoding process, before mapping these in turn to the
latent representation using a parametric function that encapsulates our desired
inductive bias(es). This allows us to impose properties like sparsity or
clustering on learned representations, and incorporate human knowledge into the
generative model. Whereas changing the prior only indirectly encourages
behavior through regularizing the encoder, InteL-VAEs are able to directly
enforce desired characteristics. Moreover, they bypass the computation and
encoder design issues caused by non-Gaussian priors, while allowing for
additional flexibility through training of the parametric mapping function. We
show that these advantages, in turn, lead to both better generative models and
better representations being learned.
