  We present a model-agnostic federated learning method for networks of
heterogeneous data and models. The network structure reflects similarities
between the (statistics of) local datasets and, in turn, their associated
local("personal") models. Our method is an instance of empirical risk
minimization, with the regularization term derived from the network structure
of data. In particular, we require well-connected local models, forming
clusters, to yield similar predictions on a common test set. The proposed
method allows for a wide range of local models. The only restriction on these
local models is that they allow for efficient implementation of regularized
empirical risk minimization (training). For a wide range of models, such
implementations are available in high-level programming libraries including
scikit-learn, Keras or PyTorch.
