  Privacy-preserving distributed processing has recently attracted considerable
attention. It aims to design solutions for conducting signal processing tasks
over networks in a decentralized fashion without violating privacy. Many
algorithms can be adopted to solve this problem such as differential privacy,
secure multiparty computation, and the recently proposed distributed
optimization based subspace perturbation. However, how these algorithms relate
to each other is not fully explored yet. In this paper, we therefore first
propose information-theoretic metrics based on mutual information. Using the
proposed metrics, we are able to compare and relate a number of existing
well-known algorithms. We then derive a lower bound on individual privacy that
gives insights on the nature of the problem. To validate the above claims, we
investigate a concrete example and compare a number of state-of-the-art
approaches in terms of different aspects such as output utility, individual
privacy and algorithm robustness against the number of corrupted parties, using
not only theoretical analysis but also numerical validation. Finally, we
discuss and provide principles for designing appropriate algorithms for
different applications.
