  Word sense induction (WSI) is the task of unsupervised clustering of word
usages within a sentence to distinguish senses. Recent work obtain strong
results by clustering lexical substitutes derived from pre-trained RNN language
models (ELMo). Adapting the method to BERT improves the scores even further. We
extend the previous method to support a dynamic rather than a fixed number of
clusters as supported by other prominent methods, and propose a method for
interpreting the resulting clusters by associating them with their most
informative substitutes. We then perform extensive error analysis revealing the
remaining sources of errors in the WSI task.
  Our code is available at https://github.com/asafamr/bertwsi.
