  Contextual embeddings represent a new generation of semantic representations
learned from Neural Language Modelling (NLM) that addresses the issue of
meaning conflation hampering traditional word embeddings. In this work, we show
that contextual embeddings can be used to achieve unprecedented gains in Word
Sense Disambiguation (WSD) tasks. Our approach focuses on creating sense-level
embeddings with full-coverage of WordNet, and without recourse to explicit
knowledge of sense distributions or task-specific modelling. As a result, a
simple Nearest Neighbors (k-NN) method using our representations is able to
consistently surpass the performance of previous systems using powerful neural
sequencing models. We also analyse the robustness of our approach when ignoring
part-of-speech and lemma features, requiring disambiguation against the full
sense inventory, and revealing shortcomings to be improved. Finally, we explore
applications of our sense embeddings for concept-level analyses of contextual
embeddings and their respective NLMs.
