  Understanding human language has been a sub-challenge on the way of
intelligent machines. The study of meaning in natural language processing (NLP)
relies on the distributional hypothesis where language elements get meaning
from the words that co-occur within contexts. The revolutionary idea of
distributed representation for a concept is close to the working of a human
mind in that the meaning of a word is spread across several neurons, and a loss
of activation will only slightly affect the memory retrieval process.
  Neural word embeddings transformed the whole field of NLP by introducing
substantial improvements in all NLP tasks. In this survey, we provide a
comprehensive literature review on neural word embeddings. We give theoretical
foundations and describe existing work by an interplay between word embeddings
and language modelling. We provide broad coverage on neural word embeddings,
including early word embeddings, embeddings targeting specific semantic
relations, sense embeddings, morpheme embeddings, and finally, contextual
representations. Finally, we describe benchmark datasets in word embeddings'
performance evaluation and downstream tasks along with the performance results
of/due to word embeddings.
