  This paper proposes a model to learn word embeddings with weighted contexts
based on part-of-speech (POS) relevance weights. POS is a fundamental element
in natural language. However, state-of-the-art word embedding models fail to
consider it. This paper proposes to use position-dependent POS relevance
weighting matrices to model the inherent syntactic relationship among words
within a context window. We utilize the POS relevance weights to model each
word-context pairs during the word embedding training process. The model
proposed in this paper paper jointly optimizes word vectors and the POS
relevance matrices. Experiments conducted on popular word analogy and word
similarity tasks all demonstrated the effectiveness of the proposed method.
