  Distributional models are derived from co-occurrences in a corpus, where only
a small proportion of all possible plausible co-occurrences will be observed.
This results in a very sparse vector space, requiring a mechanism for inferring
missing knowledge. Most methods face this challenge in ways that render the
resulting word representations uninterpretable, with the consequence that
semantic composition becomes hard to model. In this paper we explore an
alternative which involves explicitly inferring unobserved co-occurrences using
the distributional neighbourhood. We show that distributional inference
improves sparse word representations on several word similarity benchmarks and
demonstrate that our model is competitive with the state-of-the-art for
adjective-noun, noun-noun and verb-object compositions while being fully
interpretable.
