  Word embeddings are a powerful approach for unsupervised analysis of
language. Recently, Rudolph et al. (2016) developed exponential family
embeddings, which cast word embeddings in a probabilistic framework. Here, we
develop dynamic embeddings, building on exponential family embeddings to
capture how the meanings of words change over time. We use dynamic embeddings
to analyze three large collections of historical texts: the U.S. Senate
speeches from 1858 to 2009, the history of computer science ACM abstracts from
1951 to 2014, and machine learning papers on the Arxiv from 2007 to 2015. We
find dynamic embeddings provide better fits than classical embeddings and
capture interesting patterns about how language changes.
