  Federated learning (FL) that enables edge devices to collaboratively learn a
shared model while keeping their training data locally has received great
attention recently and can protect privacy in comparison with the traditional
centralized learning paradigm. However, sensitive information about the
training data can still be inferred from model parameters shared in FL.
Differential privacy (DP) is the state-of-the-art technique to defend against
those attacks. The key challenge to achieving DP in FL lies in the adverse
impact of DP noise on model accuracy, particularly for deep learning models
with large numbers of parameters. This paper develops a novel
differentially-private FL scheme named Fed-SMP that provides a client-level DP
guarantee while maintaining high model accuracy. To mitigate the impact of
privacy protection on model accuracy, Fed-SMP leverages a new technique called
Sparsified Model Perturbation (SMP) where local models are sparsified first
before being perturbed by Gaussian noise. We provide a tight end-to-end privacy
analysis for Fed-SMP using Renyi DP and prove the convergence of Fed-SMP with
both unbiased and biased sparsifications. Extensive experiments on real-world
datasets are conducted to demonstrate the effectiveness of Fed-SMP in improving
model accuracy with the same DP guarantee and saving communication cost
simultaneously.
