  Semantic representations of words have been successfully extracted from
unlabeled corpuses using neural network models like word2vec. These
representations are generally high quality and are computationally inexpensive
to train, making them popular. However, these approaches generally fail to
approximate out of vocabulary (OOV) words, a task humans can do quite easily,
using word roots and context clues. This paper proposes a neural network model
that learns high quality word representations, subword representations, and
context clue representations jointly. Learning all three types of
representations together enhances the learning of each, leading to enriched
word vectors, along with strong estimates for OOV words, via the combination of
the corresponding context clue and subword embeddings. Our model, called
Estimator Vectors (EV), learns strong word embeddings and is competitive with
state of the art methods for OOV estimation.
