  Recently, decentralized learning has emerged as a popular peer-to-peer signal
and information processing paradigm that enables model training across
geographically distributed agents in a scalable manner, without the presence of
any central server. When some of the agents are malicious (also termed as
Byzantine), resilient decentralized learning algorithms are able to limit the
impact of these Byzantine agents without knowing their number and identities,
and have guaranteed optimization errors. However, analysis of the
generalization errors, which are critical to implementations of the trained
models, is still lacking. In this paper, we provide the first analysis of the
generalization errors for a class of popular Byzantine-resilient decentralized
stochastic gradient descent (DSGD) algorithms. Our theoretical results reveal
that the generalization errors cannot be entirely eliminated because of the
presence of the Byzantine agents, even if the number of training samples are
infinitely large. Numerical experiments are conducted to confirm our
theoretical results.
