  Temporal common sense (e.g., duration and frequency of events) is crucial for
understanding natural language. However, its acquisition is challenging, partly
because such information is often not expressed explicitly in text, and human
annotation on such concepts is costly. This work proposes a novel sequence
modeling approach that exploits explicit and implicit mentions of temporal
common sense, extracted from a large corpus, to build TACOLM, a temporal common
sense language model. Our method is shown to give quality predictions of
various dimensions of temporal common sense (on UDST and a newly collected
dataset from RealNews). It also produces representations of events for relevant
tasks such as duration comparison, parent-child relations, event coreference
and temporal QA (on TimeBank, HiEVE and MCTACO) that are better than using the
standard BERT. Thus, it will be an important component of temporal NLP.
