  We present a federated learning framework that is designed to robustly
deliver good predictive performance across individual clients with
heterogeneous data. The proposed approach hinges upon a superquantile-based
learning objective that captures the tail statistics of the error distribution
over heterogeneous clients. We present a stochastic training algorithm that
interleaves differentially private client filtering with federated averaging
steps. We prove finite time convergence guarantees for the algorithm:
$O(1/\sqrt{T})$ in the nonconvex case in $T$ communication rounds and
$O(\exp(-T/\kappa^{3/2}) + \kappa/T)$ in the strongly convex case with local
condition number $\kappa$. Experimental results on benchmark datasets for
federated learning demonstrate that our approach is competitive with classical
ones in terms of average error and outperforms them in terms of tail statistics
of the error.
