  Word2vec (Mikolov et al., 2013) has proven to be successful in natural
language processing by capturing the semantic relationships between different
words. Built on top of single-word embeddings, paragraph vectors (Le and
Mikolov, 2014) find fixed-length representations for pieces of text with
arbitrary lengths, such as documents, paragraphs, and sentences. In this work,
we propose a novel interpretation for neural-network-based paragraph vectors by
developing an unsupervised generative model whose maximum likelihood solution
corresponds to traditional paragraph vectors. This probabilistic formulation
allows us to go beyond point estimates of parameters and to perform Bayesian
posterior inference. We find that the entropy of paragraph vectors decreases
with the length of documents, and that information about posterior uncertainty
improves performance in supervised learning tasks such as sentiment analysis
and paraphrase detection.
