  A major obstacle in Word Sense Disambiguation (WSD) is that word senses are
not uniformly distributed, causing existing models to generally perform poorly
on senses that are either rare or unseen during training. We propose a
bi-encoder model that independently embeds (1) the target word with its
surrounding context and (2) the dictionary definition, or gloss, of each sense.
The encoders are jointly optimized in the same representation space, so that
sense disambiguation can be performed by finding the nearest sense embedding
for each target word embedding. Our system outperforms previous
state-of-the-art models on English all-words WSD; these gains predominantly
come from improved performance on rare senses, leading to a 31.1% error
reduction on less frequent senses over prior work. This demonstrates that rare
senses can be more effectively disambiguated by modeling their definitions.
