  Self-supervised learning with masked autoencoders has recently gained
popularity for its ability to produce effective image or textual
representations, which can be applied to various downstream tasks without
retraining. However, we observe that the current masked autoencoder models lack
good generalization ability on graph data. To tackle this issue, we propose a
novel graph masked autoencoder framework called GiGaMAE. Different from
existing masked autoencoders that learn node presentations by explicitly
reconstructing the original graph components (e.g., features or edges), in this
paper, we propose to collaboratively reconstruct informative and integrated
latent embeddings. By considering embeddings encompassing graph topology and
attribute information as reconstruction targets, our model could capture more
generalized and comprehensive knowledge. Furthermore, we introduce a mutual
information based reconstruction loss that enables the effective reconstruction
of multiple targets. This learning objective allows us to differentiate between
the exclusive knowledge learned from a single target and common knowledge
shared by multiple targets. We evaluate our method on three downstream tasks
with seven datasets as benchmarks. Extensive experiments demonstrate the
superiority of GiGaMAE against state-of-the-art baselines. We hope our results
will shed light on the design of foundation models on graph-structured data.
Our code is available at: https://github.com/sycny/GiGaMAE.
