  Differential privacy (DP) has seen immense applications in learning on
tabular, image, and sequential data where instance-level privacy is concerned.
In learning on graphs, contrastingly, works on node-level privacy are highly
sparse. Challenges arise as existing DP protocols hardly apply to the
message-passing mechanism in Graph Neural Networks (GNNs).
  In this study, we propose a solution that specifically addresses the issue of
node-level privacy. Our protocol consists of two main components: 1) a sampling
routine called HeterPoisson, which employs a specialized node sampling strategy
and a series of tailored operations to generate a batch of sub-graphs with
desired properties, and 2) a randomization routine that utilizes symmetric
multivariate Laplace (SML) noise instead of the commonly used Gaussian noise.
Our privacy accounting shows this particular combination provides a non-trivial
privacy guarantee. In addition, our protocol enables GNN learning with good
performance, as demonstrated by experiments on five real-world datasets;
compared with existing baselines, our method shows significant advantages,
especially in the high privacy regime. Experimentally, we also 1) perform
membership inference attacks against our protocol and 2) apply privacy audit
techniques to confirm our protocol's privacy integrity.
  In the sequel, we present a study on a seemingly appealing approach
\cite{sajadmanesh2023gap} (USENIX'23) that protects node-level privacy via
differentially private node/instance embeddings. Unfortunately, such work has
fundamental privacy flaws, which are identified through a thorough case study.
More importantly, we prove an impossibility result of achieving both (strong)
privacy and (acceptable) utility through private instance embedding. The
implication is that such an approach has intrinsic utility barriers when
enforcing differential privacy.
