  Decentralized learning algorithms empower interconnected devices to share
data and computational resources to collaboratively train a machine learning
model without the aid of a central coordinator. In the case of heterogeneous
data distributions at the network nodes, collaboration can yield predictors
with unsatisfactory performance for a subset of the devices. For this reason,
in this work, we consider the formulation of a distributionally robust
decentralized learning task and we propose a decentralized single loop gradient
descent/ascent algorithm (AD-GDA) to directly solve the underlying minimax
optimization problem. We render our algorithm communication-efficient by
employing a compressed consensus scheme and we provide convergence guarantees
for smooth convex and non-convex loss functions. Finally, we corroborate the
theoretical findings with empirical results that highlight AD-GDA's ability to
provide unbiased predictors and to greatly improve communication efficiency
compared to existing distributionally robust algorithms.
