  Previous researches have shown that learning multiple representations for
polysemous words can improve the performance of word embeddings on many tasks.
However, this leads to another problem. Several vectors of a word may actually
point to the same meaning, namely pseudo multi-sense. In this paper, we
introduce the concept of pseudo multi-sense, and then propose an algorithm to
detect such cases. With the consideration of the detected pseudo multi-sense
cases, we try to refine the existing word embeddings to eliminate the influence
of pseudo multi-sense. Moreover, we apply our algorithm on previous released
multi-sense word embeddings and tested it on artificial word similarity tasks
and the analogy task. The result of the experiments shows that diminishing
pseudo multi-sense can improve the quality of word representations. Thus, our
method is actually an efficient way to reduce linguistic complexity.
