  Privacy has become a major concern in machine learning. In fact, the
federated learning is motivated by the privacy concern as it does not allow to
transmit the private data but only intermediate updates. However, federated
learning does not always guarantee privacy-preservation as the intermediate
updates may also reveal sensitive information. In this paper, we give an
explicit information-theoretical analysis of a federated expectation
maximization algorithm for Gaussian mixture model and prove that the
intermediate updates can cause severe privacy leakage. To address the privacy
issue, we propose a fully decentralized privacy-preserving solution, which is
able to securely compute the updates in each maximization step. Additionally,
we consider two different types of security attacks: the honest-but-curious and
eavesdropping adversary models. Numerical validation shows that the proposed
approach has superior performance compared to the existing approach in terms of
both the accuracy and privacy level.
