  Federated Learning (FL) has emerged as a prominent privacy-preserving
technique for enabling use cases like confidential clinical machine learning.
FL operates by aggregating models trained by remote devices which owns the
data. Thus, FL enables the training of powerful global models using
crowd-sourced data from a large number of learners, without compromising their
privacy. However, the aggregating server is a single point of failure when
generating the global model. Moreover, the performance of the model suffers
when the data is not independent and identically distributed (non-IID data) on
all remote devices. This leads to vastly different models being aggregated,
which can reduce the performance by as much as 50% in certain scenarios.
  In this paper, we seek to address the aforementioned issues while retaining
the benefits of FL. We propose MultiConfederated Learning: a decentralized FL
framework which is designed to handle non-IID data. Unlike traditional FL,
MultiConfederated Learning will maintain multiple models in parallel (instead
of a single global model) to help with convergence when the data is non-IID.
With the help of transfer learning, learners can converge to fewer models. In
order to increase adaptability, learners are allowed to choose which updates to
aggregate from their peers.
