  Federated graph representation learning (FedGRL) brings the benefits of
distributed training to graph structured data while simultaneously addressing
some privacy and compliance concerns related to data curation. However, several
interesting real-world graph data characteristics viz. label deficiency and
downstream task heterogeneity are not taken into consideration in current
FedGRL setups. In this paper, we consider a realistic and novel problem
setting, wherein cross-silo clients have access to vast amounts of unlabeled
data with limited or no labeled data and additionally have diverse downstream
class label domains. We then propose a novel FedGRL formulation based on model
interpolation where we aim to learn a shared global model that is optimized
collaboratively using a self-supervised objective and gets downstream task
supervision through local client models. We provide a specific instantiation of
our general formulation using BGRL a SoTA self-supervised graph representation
learning method and we empirically verify its effectiveness through realistic
cross-slio datasets: (1) we adapt the Twitch Gamer Network which naturally
simulates a cross-geo scenario and show that our formulation can provide
consistent and avg. 6.1% gains over traditional supervised federated learning
objectives and on avg. 1.7% gains compared to individual client specific
self-supervised training and (2) we construct and introduce a new cross-silo
dataset called Amazon Co-purchase Networks that have both the characteristics
of the motivated problem setting. And, we witness on avg. 11.5% gains over
traditional supervised federated learning and on avg. 1.9% gains over
individually trained self-supervised models. Both experimental results point to
the effectiveness of our proposed formulation. Finally, both our novel problem
setting and dataset contributions provide new avenues for the research in
FedGRL.
