  Most existing word embedding approaches do not distinguish the same words in
different contexts, therefore ignoring their contextual meanings. As a result,
the learned embeddings of these words are usually a mixture of multiple
meanings. In this paper, we acknowledge multiple identities of the same word in
different contexts and learn the \textbf{identity-sensitive} word embeddings.
Based on an identity-labeled text corpora, a heterogeneous network of words and
word identities is constructed to model different-levels of word
co-occurrences. The heterogeneous network is further embedded into a
low-dimensional space through a principled network embedding approach, through
which we are able to obtain the embeddings of words and the embeddings of word
identities. We study three different types of word identities including topics,
sentiments and categories. Experimental results on real-world data sets show
that the identity-sensitive word embeddings learned by our approach indeed
capture different meanings of words and outperforms competitive methods on
tasks including text classification and word similarity computation.
