  Graph contrastive learning (GCL) has become a powerful tool for learning
graph data, but its scalability remains a significant challenge. In this work,
we propose a simple yet effective training framework called Structural
Compression (StructComp) to address this issue. Inspired by a sparse low-rank
approximation on the diffusion matrix, StructComp trains the encoder with the
compressed nodes. This allows the encoder not to perform any message passing
during the training stage, and significantly reduces the number of sample pairs
in the contrastive loss. We theoretically prove that the original GCL loss can
be approximated with the contrastive loss computed by StructComp. Moreover,
StructComp can be regarded as an additional regularization term for GCL models,
resulting in a more robust encoder. Empirical studies on various datasets show
that StructComp greatly reduces the time and memory consumption while improving
model performance compared to the vanilla GCL models and scalable training
methods.
