  Majority of the text modelling techniques yield only point-estimates of
document embeddings and lack in capturing the uncertainty of the estimates.
These uncertainties give a notion of how well the embeddings represent a
document. We present Bayesian subspace multinomial model (Bayesian SMM), a
generative log-linear model that learns to represent documents in the form of
Gaussian distributions, thereby encoding the uncertainty in its co-variance.
Additionally, in the proposed Bayesian SMM, we address a commonly encountered
problem of intractability that appears during variational inference in
mixed-logit models. We also present a generative Gaussian linear classifier for
topic identification that exploits the uncertainty in document embeddings. Our
intrinsic evaluation using perplexity measure shows that the proposed Bayesian
SMM fits the data better as compared to the state-of-the-art neural variational
document model on Fisher speech and 20Newsgroups text corpora. Our topic
identification experiments show that the proposed systems are robust to
over-fitting on unseen test data. The topic ID results show that the proposed
model is outperforms state-of-the-art unsupervised topic models and achieve
comparable results to the state-of-the-art fully supervised discriminative
models.
