  Distributed word embeddings such as Word2Vec and GloVe have been widely
adopted in industrial context settings. Major technical applications of GloVe
include recommender systems and natural language processing. The fundamental
theory behind GloVe relies on the selection of a weighting function in the
weighted least squres formulation that computes the powered ratio of word
occurrence count and the maximum word count in the corpus. However, the initial
formulation of GloVe is not theoretically sound in two aspects, namely the
selection of the weighting function and its power exponent is ad-hoc. In this
paper, we utilize the theory of extreme value analysis and propose a
theoretically accurate version of GloVe. By reformulating the weighted least
squares loss function as the expected loss function and accurately choosing the
power exponent, we create a theoretically accurate version of GloVe. We
demonstrate the competitiveness of our algorithm and show that the initial
formulation of GloVe with the suggested optimal parameter can be viewed as a
special case of our paradigm.
