  Federated Learning (FL) with Secure Aggregation (SA) has gained significant
attention as a privacy preserving framework for training machine learning
models while preventing the server from learning information about users' data
from their individual encrypted model updates. Recent research has extended
privacy guarantees of FL with SA by bounding the information leakage through
the aggregate model over multiple training rounds thanks to leveraging the
"noise" from other users' updates. However, the privacy metric used in that
work (mutual information) measures the on-average privacy leakage, without
providing any privacy guarantees for worse-case scenarios. To address this, in
this work we study the conditions under which FL with SA can provide worst-case
differential privacy guarantees. Specifically, we formally identify the
necessary condition that SA can provide DP without addition noise. We then
prove that when the randomness inside the aggregated model update is Gaussian
with non-singular covariance matrix, SA can provide differential privacy
guarantees with the level of privacy $\epsilon$ bounded by the reciprocal of
the minimum eigenvalue of the covariance matrix. However, we further
demonstrate that in practice, these conditions are almost unlikely to hold and
hence additional noise added in model updates is still required in order for SA
in FL to achieve DP. Lastly, we discuss the potential solution of leveraging
inherent randomness inside aggregated model update to reduce the amount of
addition noise required for DP guarantee.
