  Graph Contrastive Learning (GCL) has recently drawn much research interest
for learning generalizable node representations in a self-supervised manner. In
general, the contrastive learning process in GCL is performed on top of the
representations learned by a graph neural network (GNN) backbone, which
transforms and propagates the node contextual information based on its local
neighborhoods. However, nodes sharing similar characteristics may not always be
geographically close, which poses a great challenge for unsupervised GCL
efforts due to their inherent limitations in capturing such global graph
knowledge. In this work, we address their inherent limitations by proposing a
simple yet effective framework -- Simple Neural Networks with Structural and
Semantic Contrastive Learning} (S^3-CL). Notably, by virtue of the proposed
structural and semantic contrastive learning algorithms, even a simple neural
network can learn expressive node representations that preserve valuable global
structural and semantic patterns. Our experiments demonstrate that the node
representations learned by S^3-CL achieve superior performance on different
downstream tasks compared with the state-of-the-art unsupervised GCL methods.
Implementation and more experimental details are publicly available at
\url{https://github.com/kaize0409/S-3-CL.}
