  We investigate the robustness and security issues from a novel and practical
setting: a group of malicious clients has impacted the model during training by
disguising their identities and acting as benign clients, and only revealing
their adversary position after the training to conduct transferable adversarial
attacks with their data, which is usually a subset of the data that FL system
is trained with. Our aim is to offer a full understanding of the challenges the
FL system faces in this practical setting across a spectrum of configurations.
We notice that such an attack is possible, but the federated model is more
robust compared with its centralized counterpart when the accuracy on clean
images is comparable. Through our study, we hypothesized the robustness is from
two factors: the decentralized training on distributed data and the averaging
operation. We provide evidence from both the perspective of empirical
experiments and theoretical analysis. Our work has implications for
understanding the robustness of federated learning systems and poses a
practical question for federated learning applications.
