  Recently, Graph Neural Networks (GNNs) have been receiving a spotlight as a
powerful tool that can effectively serve various inference tasks on graph
structured data. As the size of real-world graphs continues to scale, the GNN
training system faces a scalability challenge. Distributed training is a
popular approach to address this challenge by scaling out CPU nodes. However,
not much attention has been paid to disk-based GNN training, which can scale up
the single-node system in a more cost-effective manner by leveraging
high-performance storage devices like NVMe SSDs. We observe that the data
movement between the main memory and the disk is the primary bottleneck in the
SSD-based training system, and that the conventional GNN training pipeline is
sub-optimal without taking this overhead into account. Thus, we propose Ginex,
the first SSD-based GNN training system that can process billion-scale graph
datasets on a single machine. Inspired by the inspector-executor execution
model in compiler optimization, Ginex restructures the GNN training pipeline by
separating sample and gather stages. This separation enables Ginex to realize a
provably optimal replacement algorithm, known as Belady's algorithm, for
caching feature vectors in memory, which account for the dominant portion of
I/O accesses. According to our evaluation with four billion-scale graph
datasets, Ginex achieves 2.11x higher training throughput on average (up to
2.67x at maximum) than the SSD-extended PyTorch Geometric.
