  Unsupervised/self-supervised pre-training methods for graph representation
learning have recently attracted increasing research interests, and they are
shown to be able to generalize to various downstream applications. Yet, the
adversarial robustness of such pre-trained graph learning models remains
largely unexplored. More importantly, most existing defense techniques designed
for end-to-end graph representation learning methods require pre-specified
label definitions, and thus cannot be directly applied to the pre-training
methods. In this paper, we propose an unsupervised defense technique to
robustify pre-trained deep graph models, so that the perturbations on the input
graph can be successfully identified and blocked before the model is applied to
different downstream tasks. Specifically, we introduce a mutual
information-based measure, \textit{graph representation vulnerability (GRV)},
to quantify the robustness of graph encoders on the representation space. We
then formulate an optimization problem to learn the graph representation by
carefully balancing the trade-off between the expressive power and the
robustness (\emph{i.e.}, GRV) of the graph encoder. The discrete nature of
graph topology and the joint space of graph data make the optimization problem
intractable to solve. To handle the above difficulty and to reduce
computational expense, we further relax the problem and thus provide an
approximate solution. Additionally, we explore a provable connection between
the robustness of the unsupervised graph encoder and that of models on
downstream tasks. Extensive experiments demonstrate that even without access to
labels and tasks, our model is still able to enhance robustness against
adversarial attacks on three downstream tasks (node classification, link
prediction, and community detection) by an average of +16.5% compared with
existing methods.
