  Since conventional knowledge embedding models cannot take full advantage of
the abundant textual information, there have been extensive research efforts in
enhancing knowledge embedding using texts. However, existing enhancement
approaches cannot apply to temporal knowledge graphs (tKGs), which contain
time-dependent event knowledge with complex temporal dynamics. Specifically,
existing enhancement approaches often assume knowledge embedding is
time-independent. In contrast, the entity embedding in tKG models usually
evolves, which poses the challenge of aligning temporally relevant texts with
entities. To this end, we propose to study enhancing temporal knowledge
embedding with textual data in this paper. As an approach to this task, we
propose Enhanced Temporal Knowledge Embeddings with Contextualized Language
Representations (ECOLA), which takes the temporal aspect into account and
injects textual information into temporal knowledge embedding. To evaluate
ECOLA, we introduce three new datasets for training and evaluating ECOLA.
Extensive experiments show that ECOLA significantly enhances temporal KG
embedding models with up to 287% relative improvements regarding Hits@1 on the
link prediction task. The code and models are publicly available on
https://anonymous.4open.science/r/ECOLA.
