  Graph neural networks (GNNs) are naturally distributed architectures for
learning representations from network data. This renders them suitable
candidates for decentralized tasks. In these scenarios, the underlying graph
often changes with time due to link failures or topology variations, creating a
mismatch between the graphs on which GNNs were trained and the ones on which
they are tested. Online learning can be leveraged to retrain GNNs at testing
time to overcome this issue. However, most online algorithms are centralized
and usually offer guarantees only on convex problems, which GNNs rarely lead
to. This paper develops the Wide and Deep GNN (WD-GNN), a novel architecture
that can be updated with distributed online learning mechanisms. The WD-GNN
consists of two components: the wide part is a linear graph filter and the deep
part is a nonlinear GNN. At training time, the joint wide and deep architecture
learns nonlinear representations from data. At testing time, the wide, linear
part is retrained, while the deep, nonlinear one remains fixed. This often
leads to a convex formulation. We further propose a distributed online learning
algorithm that can be implemented in a decentralized setting. We also show the
stability of the WD-GNN to changes of the underlying graph and analyze the
convergence of the proposed online learning procedure. Experiments on movie
recommendation, source localization and robot swarm control corroborate
theoretical findings and show the potential of the WD-GNN for distributed
online learning.
