  Federated learning, while being a promising approach for collaborative model
training, is susceptible to poisoning attacks due to its decentralized nature.
Backdoor attacks, in particular, have shown remarkable stealthiness, as they
selectively compromise predictions for inputs containing triggers. Previous
endeavors to detect and mitigate such attacks are based on the Independent and
Identically Distributed (IID) data assumption where benign model updates
exhibit high-level similarity in multiple feature spaces due to IID data. Thus,
outliers are detected as backdoor attacks. Nevertheless, non-IID data presents
substantial challenges in backdoor attack detection, as the data variety
introduces variance among benign models, making outlier detection-based
mechanisms less effective.
  We propose a novel distribution-aware anomaly detection mechanism, BoBa, to
address this problem. In order to differentiate outliers arising from data
variety versus backdoor attack, we propose to break down the problem into two
steps: clustering clients utilizing their data distribution followed by a
voting-based detection. Based on the intuition that clustering and subsequent
backdoor detection can drastically benefit from knowing client data
distributions, we propose a novel data distribution inference mechanism. To
improve detection robustness, we introduce an overlapping clustering method,
where each client is associated with multiple clusters, ensuring that the
trustworthiness of a model update is assessed collectively by multiple clusters
rather than a single cluster. Through extensive evaluations, we demonstrate
that BoBa can reduce the attack success rate to lower than 0.001 while
maintaining high main task accuracy across various attack strategies and
experimental settings.
