  Federated learning poses new statistical and systems challenges in training
machine learning models over distributed networks of devices. In this work, we
show that multi-task learning is naturally suited to handle the statistical
challenges of this setting, and propose a novel systems-aware optimization
method, MOCHA, that is robust to practical systems issues. Our method and
theory for the first time consider issues of high communication cost,
stragglers, and fault tolerance for distributed multi-task learning. The
resulting method achieves significant speedups compared to alternatives in the
federated setting, as we demonstrate through simulations on real-world
federated datasets.
