  In this paper, we study the challenging task of Byzantine-robust
decentralized training on arbitrary communication graphs. Unlike federated
learning where workers communicate through a server, workers in the
decentralized environment can only talk to their neighbors, making it harder to
reach consensus and benefit from collaborative training. To address these
issues, we propose a ClippedGossip algorithm for Byzantine-robust consensus and
optimization, which is the first to provably converge to a
$O(\delta_{\max}\zeta^2/\gamma^2)$ neighborhood of the stationary point for
non-convex objectives under standard assumptions. Finally, we demonstrate the
encouraging empirical performance of ClippedGossip under a large number of
attacks.
