  While word embeddings are currently predominant for natural language
processing, most of existing models learn them solely from their contexts.
However, these context-based word embeddings are limited since not all words'
meaning can be learned based on only context. Moreover, it is also difficult to
learn the representation of the rare words due to data sparsity problem. In
this work, we address these issues by learning the representations of words by
integrating their intrinsic (descriptive) and extrinsic (contextual)
information. To prove the effectiveness of our model, we evaluate it on four
tasks, including word similarity, reverse dictionaries,Wiki link prediction,
and document classification. Experiment results show that our model is powerful
in both word and document modeling.
