  Current work in lexical distributed representations maps each word to a point
vector in low-dimensional space. Mapping instead to a density provides many
interesting advantages, including better capturing uncertainty about a
representation and its relationships, expressing asymmetries more naturally
than dot product or cosine similarity, and enabling more expressive
parameterization of decision boundaries. This paper advocates for density-based
distributed embeddings and presents a method for learning representations in
the space of Gaussian distributions. We compare performance on various word
embedding benchmarks, investigate the ability of these embeddings to model
entailment and other asymmetric relationships, and explore novel properties of
the representation.
