  Graph neural networks (GNNs) have been demonstrated to be a powerful
algorithmic model in broad application fields for their effectiveness in
learning over graphs. To scale GNN training up for large-scale and ever-growing
graphs, the most promising solution is distributed training which distributes
the workload of training across multiple computing nodes. At present, the
volume of related research on distributed GNN training is exceptionally vast,
accompanied by an extraordinarily rapid pace of publication. Moreover, the
approaches reported in these studies exhibit significant divergence. This
situation poses a considerable challenge for newcomers, hindering their ability
to grasp a comprehensive understanding of the workflows, computational
patterns, communication strategies, and optimization techniques employed in
distributed GNN training. As a result, there is a pressing need for a survey to
provide correct recognition, analysis, and comparisons in this field. In this
paper, we provide a comprehensive survey of distributed GNN training by
investigating various optimization techniques used in distributed GNN training.
First, distributed GNN training is classified into several categories according
to their workflows. In addition, their computational patterns and communication
patterns, as well as the optimization techniques proposed by recent work are
introduced. Second, the software frameworks and hardware platforms of
distributed GNN training are also introduced for a deeper understanding. Third,
distributed GNN training is compared with distributed training of deep neural
networks, emphasizing the uniqueness of distributed GNN training. Finally,
interesting issues and opportunities in this field are discussed.
