  For text analysis, one often resorts to a lossy representation that either
completely ignores word order or embeds each word as a low-dimensional dense
feature vector. In this paper, we propose convolutional Poisson factor analysis
(CPFA) that directly operates on a lossless representation that processes the
words in each document as a sequence of high-dimensional one-hot vectors. To
boost its performance, we further propose the convolutional Poisson gamma
belief network (CPGBN) that couples CPFA with the gamma belief network via a
novel probabilistic pooling layer. CPFA forms words into phrases and captures
very specific phrase-level topics, and CPGBN further builds a hierarchy of
increasingly more general phrase-level topics. For efficient inference, we
develop both a Gibbs sampler and a Weibull distribution based convolutional
variational auto-encoder. Experimental results demonstrate that CPGBN can
extract high-quality text latent representations that capture the word order
information, and hence can be leveraged as a building block to enrich a wide
variety of existing latent variable models that ignore word order.
