  Classification tasks on labeled graph-structured data have many important
applications ranging from social recommendation to financial modeling. Deep
neural networks are increasingly being used for node classification on graphs,
wherein nodes with similar features have to be given the same label. Graph
convolutional networks (GCNs) are one such widely studied neural network
architecture that perform well on this task. However, powerful link-stealing
attacks on GCNs have recently shown that even with black-box access to the
trained model, inferring which links (or edges) are present in the training
graph is practical. In this paper, we present a new neural network architecture
called LPGNet for training on graphs with privacy-sensitive edges. LPGNet
provides differential privacy (DP) guarantees for edges using a novel design
for how graph edge structure is used during training. We empirically show that
LPGNet models often lie in the sweet spot between providing privacy and
utility: They can offer better utility than "trivially" private architectures
which use no edge information (e.g., vanilla MLPs) and better resilience
against existing link-stealing attacks than vanilla GCNs which use the full
edge structure. LPGNet also offers consistently better privacy-utility
tradeoffs than DPGCN, which is the state-of-the-art mechanism for retrofitting
differential privacy into conventional GCNs, in most of our evaluated datasets.
