  Pre-training GNNs to extract transferable knowledge and apply it to
downstream tasks has become the de facto standard of graph representation
learning. Recent works focused on designing self-supervised pre-training tasks
to extract useful and universal transferable knowledge from large-scale
unlabeled data. However, they have to face an inevitable question: traditional
pre-training strategies that aim at extracting useful information about
pre-training tasks, may not extract all useful information about the downstream
task. In this paper, we reexamine the pre-training process within traditional
pre-training and fine-tuning frameworks from the perspective of Information
Bottleneck (IB) and confirm that the forgetting phenomenon in pre-training
phase may cause detrimental effects on downstream tasks. Therefore, we propose
a novel \underline{D}elayed \underline{B}ottlenecking \underline{P}re-training
(DBP) framework which maintains as much as possible mutual information between
latent representations and training data during pre-training phase by
suppressing the compression operation and delays the compression operation to
fine-tuning phase to make sure the compression can be guided with labeled
fine-tuning data and downstream tasks. To achieve this, we design two
information control objectives that can be directly optimized and further
integrate them into the actual model design. Extensive experiments on both
chemistry and biology domains demonstrate the effectiveness of DBP.
