  Federated learning is a promising framework for learning over decentralized
data spanning multiple regions. This approach avoids expensive central training
data aggregation cost and can improve privacy because distributed sites do not
have to reveal privacy-sensitive data. In this paper, we develop a
privacy-preserving decentralized aggregation protocol for federated learning.
We formulate the distributed aggregation protocol with the Alternating
Direction Method of Multiplier (ADMM) and examine its privacy weakness. Unlike
prior work that use Differential Privacy or homomorphic encryption for privacy,
we develop a protocol that controls communication among participants in each
round of aggregation to minimize privacy leakage. We establish its privacy
guarantee against an honest-but-curious adversary. We also propose an efficient
algorithm to construct such a communication pattern, inspired by combinatorial
block design theory. Our secure aggregation protocol based on this novel group
communication pattern design leads to an efficient algorithm for federated
training with privacy guarantees. We evaluate our federated training algorithm
on image classification and next-word prediction applications over benchmark
datasets with 9 and 15 distributed sites. Evaluation results show that our
algorithm performs comparably to the standard centralized federated learning
method while preserving privacy; the degradation in test accuracy is only up to
0.73%.
