  Graph neural network (GNN) has been demonstrated to be a powerful model in
many domains for its effectiveness in learning over graphs. To scale GNN
training for large graphs, a widely adopted approach is distributed training
which accelerates training using multiple computing nodes. Maximizing the
performance is essential, but the execution of distributed GNN training remains
preliminarily understood. In this work, we provide an in-depth analysis of
distributed GNN training on GPUs, revealing several significant observations
and providing useful guidelines for both software optimization and hardware
optimization.
