  Federated learning has emerged as a promising paradigm for collaborative
model training while preserving data privacy. However, recent studies have
shown that it is vulnerable to various privacy attacks, such as data
reconstruction attacks. In this paper, we provide a theoretical analysis of
privacy leakage in federated learning from two perspectives: linear algebra and
optimization theory. From the linear algebra perspective, we prove that when
the Jacobian matrix of the batch data is not full rank, there exist different
batches of data that produce the same model update, thereby ensuring a level of
privacy. We derive a sufficient condition on the batch size to prevent data
reconstruction attacks. From the optimization theory perspective, we establish
an upper bound on the privacy leakage in terms of the batch size, the
distortion extent, and several other factors. Our analysis provides insights
into the relationship between privacy leakage and various aspects of federated
learning, offering a theoretical foundation for designing privacy-preserving
federated learning algorithms.
