  We propose a new method that leverages contextual embeddings for the task of
diachronic semantic shift detection by generating time specific word
representations from BERT embeddings. The results of our experiments in the
domain specific LiverpoolFC corpus suggest that the proposed method has
performance comparable to the current state-of-the-art without requiring any
time consuming domain adaptation on large corpora. The results on the newly
created Brexit news corpus suggest that the method can be successfully used for
the detection of a short-term yearly semantic shift. And lastly, the model also
shows promising results in a multilingual settings, where the task was to
detect differences and similarities between diachronic semantic shifts in
different languages.
