  Pre-training on graph neural networks (GNNs) aims to learn transferable
knowledge for downstream tasks with unlabeled data, and it has recently become
an active research area. The success of graph pre-training models is often
attributed to the massive amount of input data. In this paper, however, we
identify the curse of big data phenomenon in graph pre-training: more training
data do not necessarily lead to better downstream performance. Motivated by
this observation, we propose a better-with-less framework for graph
pre-training: fewer, but carefully chosen data are fed into a GNN model to
enhance pre-training. The proposed pre-training pipeline is called the
data-active graph pre-training (APT) framework, and is composed of a graph
selector and a pre-training model. The graph selector chooses the most
representative and instructive data points based on the inherent properties of
graphs as well as predictive uncertainty. The proposed predictive uncertainty,
as feedback from the pre-training model, measures the confidence level of the
model in the data. When fed with the chosen data, on the other hand, the
pre-training model grasps an initial understanding of the new, unseen data, and
at the same time attempts to remember the knowledge learned from previous data.
Therefore, the integration and interaction between these two components form a
unified framework (APT), in which graph pre-training is performed in a
progressive and iterative way. Experiment results show that the proposed APT is
able to obtain an efficient pre-training model with fewer training data and
better downstream performance.
