  In federated learning, models are learned from users' data that are held
private in their edge devices, by aggregating them in the service provider's
"cloud" to obtain a global model. Such global model is of great commercial
value in, e.g., improving the customers' experience. In this paper we focus on
two possible areas of improvement of the state of the art. First, we take the
difference between user habits into account and propose a quadratic
penalty-based formulation, for efficient learning of the global model that
allows to personalize local models. Second, we address the latency issue
associated with the heterogeneous training time on edge devices, by exploiting
a hierarchical structure modeling communication not only between the cloud and
edge devices, but also within the cloud. Specifically, we devise a tailored
block coordinate descent-based computation scheme, accompanied with
communication protocols for both the synchronous and asynchronous cloud
settings. We characterize the theoretical convergence rate of the algorithm,
and provide a variant that performs empirically better. We also prove that the
asynchronous protocol, inspired by multi-agent consensus technique, has the
potential for large gains in latency compared to a synchronous setting when the
edge-device updates are intermittent. Finally, experimental results are
provided that corroborate not only the theory, but also show that the system
leads to faster convergence for personalized models on the edge devices,
compared to the state of the art.
