  In recent years a certain success in the task of modeling lexical semantics
was obtained with distributional semantic models. Nevertheless, the scientific
community is still unaware what is the most reliable evaluation method for
these models. Some researchers argue that the only possible gold standard could
be obtained from neuro-cognitive resources that store information about human
cognition. One of such resources is eye movement data on silent reading. The
goal of this work is to test the hypothesis of whether such data could be used
to evaluate distributional semantic models on different languages. We propose
experiments with English and Russian eye movement datasets (Provo Corpus, GECO
and Russian Sentence Corpus), word vectors (Skip-Gram models trained on
national corpora and Web corpora) and word similarity datasets of Russian and
English assessed by humans in order to find the existence of correlation
between embeddings and eye movement data and test the hypothesis that this
correlation is language independent. As a result, we found that the validity of
the hypothesis being tested could be questioned.
