  Given the input graph and its label/property, several key problems of graph
learning, such as finding interpretable subgraphs, graph denoising and graph
compression, can be attributed to the fundamental problem of recognizing a
subgraph of the original one. This subgraph shall be as informative as
possible, yet contains less redundant and noisy structure. This problem setting
is closely related to the well-known information bottleneck (IB) principle,
which, however, has less been studied for the irregular graph data and graph
neural networks (GNNs). In this paper, we propose a framework of Graph
Information Bottleneck (GIB) for the subgraph recognition problem in deep graph
learning. Under this framework, one can recognize the maximally informative yet
compressive subgraph, named IB-subgraph. However, the GIB objective is
notoriously hard to optimize, mostly due to the intractability of the mutual
information of irregular graph data and the unstable optimization process. In
order to tackle these challenges, we propose: i) a GIB objective based-on a
mutual information estimator for the irregular graph data; ii) a bi-level
optimization scheme to maximize the GIB objective; iii) a connectivity loss to
stabilize the optimization process. We evaluate the properties of the
IB-subgraph in three application scenarios: improvement of graph
classification, graph interpretation and graph denoising. Extensive experiments
demonstrate that the information-theoretic IB-subgraph enjoys superior graph
properties.
