  Many real world graphs contain time domain information. Temporal Graph Neural
Networks capture temporal information as well as structural and contextual
information in the generated dynamic node embeddings. Researchers have shown
that these embeddings achieve state-of-the-art performance in many different
tasks. In this work, we propose TGL, a unified framework for large-scale
offline Temporal Graph Neural Network training where users can compose various
Temporal Graph Neural Networks with simple configuration files. TGL comprises
five main components, a temporal sampler, a mailbox, a node memory module, a
memory updater, and a message passing engine. We design a Temporal-CSR data
structure and a parallel sampler to efficiently sample temporal neighbors to
formtraining mini-batches. We propose a novel random chunk scheduling technique
that mitigates the problem of obsolete node memory when training with a large
batch size. To address the limitations of current TGNNs only being evaluated on
small-scale datasets, we introduce two large-scale real-world datasets with 0.2
and 1.3 billion temporal edges. We evaluate the performance of TGL on four
small-scale datasets with a single GPU and the two large datasets with multiple
GPUs for both link prediction and node classification tasks. We compare TGL
with the open-sourced code of five methods and show that TGL achieves similar
or better accuracy with an average of 13x speedup. Our temporal parallel
sampler achieves an average of 173x speedup on a multi-core CPU compared with
the baselines. On a 4-GPU machine, TGL can train one epoch of more than one
billion temporal edges within 1-10 hours. To the best of our knowledge, this is
the first work that proposes a general framework for large-scale Temporal Graph
Neural Networks training on multiple GPUs.
