  This paper shows that, modern word embeddings contain information that
distinguishes synonyms and antonyms despite small cosine similarities between
corresponding vectors. This information is encoded in the geometry of the
embeddings and could be extracted with a straight-forward and intuitive
manifold learning procedure or a contrasting map. Such a map is trained on a
small labeled subset of the data and can produce new embeddings that explicitly
highlight specific semantic attributes of the word. The new embeddings produced
by the map are shown to improve the performance on downstream tasks.
