  In the context of natural language processing, representation learning has
emerged as a newly active research subject because of its excellent performance
in many applications. Learning representations of words is a pioneering study
in this school of research. However, paragraph (or sentence and document)
embedding learning is more suitable/reasonable for some tasks, such as
sentiment classification and document summarization. Nevertheless, as far as we
are aware, there is relatively less work focusing on the development of
unsupervised paragraph embedding methods. Classic paragraph embedding methods
infer the representation of a given paragraph by considering all of the words
occurring in the paragraph. Consequently, those stop or function words that
occur frequently may mislead the embedding learning process to produce a misty
paragraph representation. Motivated by these observations, our major
contributions in this paper are twofold. First, we propose a novel unsupervised
paragraph embedding method, named the essence vector (EV) model, which aims at
not only distilling the most representative information from a paragraph but
also excluding the general background information to produce a more informative
low-dimensional vector representation for the paragraph. Second, in view of the
increasing importance of spoken content processing, an extension of the EV
model, named the denoising essence vector (D-EV) model, is proposed. The D-EV
model not only inherits the advantages of the EV model but also can infer a
more robust representation for a given spoken paragraph against imperfect
speech recognition.
