  Learning representations of words in a continuous space is perhaps the most
fundamental task in NLP, however words interact in ways much richer than vector
dot product similarity can provide. Many relationships between words can be
expressed set-theoretically, for example, adjective-noun compounds (eg. "red
cars"$\subseteq$"cars") and homographs (eg. "tongue"$\cap$"body" should be
similar to "mouth", while "tongue"$\cap$"language" should be similar to
"dialect") have natural set-theoretic interpretations. Box embeddings are a
novel region-based representation which provide the capability to perform these
set-theoretic operations. In this work, we provide a fuzzy-set interpretation
of box embeddings, and learn box representations of words using a set-theoretic
training objective. We demonstrate improved performance on various word
similarity tasks, particularly on less common words, and perform a quantitative
and qualitative analysis exploring the additional unique expressivity provided
by Word2Box.
