  Graph Neural Networks (GNNs) have been successfully applied to many
real-world static graphs. However, the success of static graphs has not fully
translated to dynamic graphs due to the limitations in model design, evaluation
settings, and training strategies. Concretely, existing dynamic GNNs do not
incorporate state-of-the-art designs from static GNNs, which limits their
performance. Current evaluation settings for dynamic GNNs do not fully reflect
the evolving nature of dynamic graphs. Finally, commonly used training methods
for dynamic GNNs are not scalable. Here we propose ROLAND, an effective graph
representation learning framework for real-world dynamic graphs. At its core,
the ROLAND framework can help researchers easily repurpose any static GNN to
dynamic graphs. Our insight is to view the node embeddings at different GNN
layers as hierarchical node states and then recurrently update them over time.
We then introduce a live-update evaluation setting for dynamic graphs that
mimics real-world use cases, where GNNs are making predictions and being
updated on a rolling basis. Finally, we propose a scalable and efficient
training approach for dynamic GNNs via incremental training and meta-learning.
We conduct experiments over eight different dynamic graph datasets on future
link prediction tasks. Models built using the ROLAND framework achieve on
average 62.7% relative mean reciprocal rank (MRR) improvement over
state-of-the-art baselines under the standard evaluation settings on three
datasets. We find state-of-the-art baselines experience out-of-memory errors
for larger datasets, while ROLAND can easily scale to dynamic graphs with 56
million edges. After re-implementing these baselines using the ROLAND training
strategy, ROLAND models still achieve on average 15.5% relative MRR improvement
over the baselines.
