  Compared with word embedding based on point representation,
distribution-based word embedding shows more flexibility in expressing
uncertainty and therefore embeds richer semantic information when representing
words. The Wasserstein distance provides a natural notion of dissimilarity with
probability measures and has a closed-form solution when measuring the distance
between two Gaussian distributions. Therefore, with the aim of representing
words in a highly efficient way, we propose to operate a Gaussian word
embedding model with a loss function based on the Wasserstein distance. Also,
external information from ConceptNet will be used to semi-supervise the results
of the Gaussian word embedding. Thirteen datasets from the word similarity
task, together with one from the word entailment task, and six datasets from
the downstream document classification task will be evaluated in this paper to
test our hypothesis.
