  This paper proposes Dirichlet Variational Autoencoder (DirVAE) using a
Dirichlet prior for a continuous latent variable that exhibits the
characteristic of the categorical probabilities. To infer the parameters of
DirVAE, we utilize the stochastic gradient method by approximating the Gamma
distribution, which is a component of the Dirichlet distribution, with the
inverse Gamma CDF approximation. Additionally, we reshape the component
collapsing issue by investigating two problem sources, which are decoder weight
collapsing and latent value collapsing, and we show that DirVAE has no
component collapsing; while Gaussian VAE exhibits the decoder weight collapsing
and Stick-Breaking VAE shows the latent value collapsing. The experimental
results show that 1) DirVAE models the latent representation result with the
best log-likelihood compared to the baselines; and 2) DirVAE produces more
interpretable latent values with no collapsing issues which the baseline models
suffer from. Also, we show that the learned latent representation from the
DirVAE achieves the best classification accuracy in the semi-supervised and the
supervised classification tasks on MNIST, OMNIGLOT, and SVHN compared to the
baseline VAEs. Finally, we demonstrated that the DirVAE augmented topic models
show better performances in most cases.
