  Continual learning on graph data has recently attracted paramount attention
for its aim to resolve the catastrophic forgetting problem on existing tasks
while adapting the sequentially updated model to newly emerged graph tasks.
While there have been efforts to summarize progress on continual learning
research over Euclidean data, e.g., images and texts, a systematic review of
progress in continual learning on graphs, a.k.a, continual graph learning (CGL)
or lifelong graph learning, is still demanding. Graph data are far more complex
in terms of data structures and application scenarios, making CGL task
settings, model designs, and applications extremely challenging. To bridge the
gap, we provide a comprehensive review of existing continual graph learning
(CGL) algorithms by elucidating the different task settings and categorizing
the existing methods based on their characteristics. We compare the CGL methods
with traditional continual learning techniques and analyze the applicability of
the traditional continual learning techniques to CGL tasks. Additionally, we
review the benchmark works that are crucial to CGL research. Finally, we
discuss the remaining challenges and propose several future directions. We will
maintain an up-to-date GitHub repository featuring a comprehensive list of CGL
algorithms, accessible at
https://github.com/UConn-DSIS/Survey-of-Continual-Learning-on-Graphs.
