  To build a flexible and interpretable model for document analysis, we develop
deep autoencoding topic model (DATM) that uses a hierarchy of gamma
distributions to construct its multi-stochastic-layer generative network. In
order to provide scalable posterior inference for the parameters of the
generative network, we develop topic-layer-adaptive stochastic gradient
Riemannian MCMC that jointly learns simplex-constrained global parameters
across all layers and topics, with topic and layer specific learning rates.
Given a posterior sample of the global parameters, in order to efficiently
infer the local latent representations of a document under DATM across all
stochastic layers, we propose a Weibull upward-downward variational encoder
that deterministically propagates information upward via a deep neural network,
followed by a Weibull distribution based stochastic downward generative model.
To jointly model documents and their associated labels, we further propose
supervised DATM that enhances the discriminative power of its latent
representations. The efficacy and scalability of our models are demonstrated on
both unsupervised and supervised learning tasks on big corpora.
