  We train a diachronic long short-term memory (LSTM) part-of-speech tagger on
a large corpus of American English from the 19th, 20th, and 21st centuries. We
analyze the tagger's ability to implicitly learn temporal structure between
years, and the extent to which this knowledge can be transferred to date new
sentences. The learned year embeddings show a strong linear correlation between
their first principal component and time. We show that temporal information
encoded in the model can be used to predict novel sentences' years of
composition relatively well. Comparisons to a feedforward baseline suggest that
the temporal change learned by the LSTM is syntactic rather than purely
lexical. Thus, our results suggest that our tagger is implicitly learning to
model syntactic change in American English over the course of the 19th, 20th,
and early 21st centuries.
