  Word embedding has become ubiquitous and is widely used in various natural
language processing (NLP) tasks, such as web retrieval, web semantic analysis,
and machine translation, and so on. Unfortunately, training the word embedding
in a relatively large corpus is prohibitively expensive. We propose a
graph-based word embedding algorithm, called Word-Graph2vec, which converts the
large corpus into a word co-occurrence graph, then takes the word sequence
samples from this graph by randomly traveling and trains the word embedding on
this sampling corpus in the end. We posit that because of the limited
vocabulary, huge idioms, and fixed expressions in English, the size and density
of the word co-occurrence graph change slightly with the increase in the
training corpus. So that Word-Graph2vec has stable runtime on the large-scale
data set, and its performance advantage becomes more and more obvious with the
growth of the training corpus. Extensive experiments conducted on real-world
datasets show that the proposed algorithm outperforms traditional Word2vec four
to five times in terms of efficiency and two to three times than FastText,
while the error generated by the random walk technique is small.
