  In this paper we present a model for unsupervised topic discovery in texts
corpora. The proposed model uses documents, words, and topics lookup table
embedding as neural network model parameters to build probabilities of words
given topics, and probabilities of topics given documents. These probabilities
are used to recover by marginalization probabilities of words given documents.
For very large corpora where the number of documents can be in the order of
billions, using a neural auto-encoder based document embedding is more scalable
then using a lookup table embedding as classically done. We thus extended the
lookup based document embedding model to continuous auto-encoder based model.
Our models are trained using probabilistic latent semantic analysis (PLSA)
assumptions. We evaluated our models on six datasets with a rich variety of
contents. Conducted experiments demonstrate that the proposed neural topic
models are very effective in capturing relevant topics. Furthermore,
considering perplexity metric, conducted evaluation benchmarks show that our
topic models outperform latent Dirichlet allocation (LDA) model which is
classically used to address topic discovery tasks.
