  We study distributed estimation and learning problems in a networked
environment where agents exchange information to estimate unknown statistical
properties of random variables from their privately observed samples. The
agents can collectively estimate the unknown quantities by exchanging
information about their private observations, but they also face privacy risks.
Our novel algorithms extend the existing distributed estimation literature and
enable the participating agents to estimate a complete sufficient statistic
from private signals acquired offline or online over time and to preserve the
privacy of their signals and network neighborhoods. This is achieved through
linear aggregation schemes with adjusted randomization schemes that add noise
to the exchanged estimates subject to differential privacy (DP) constraints,
both in an offline and online manner. We provide convergence rate analysis and
tight finite-time convergence bounds. We show that the noise that minimizes the
convergence time to the best estimates is the Laplace noise, with parameters
corresponding to each agent's sensitivity to their signal and network
characteristics. Our algorithms are amenable to dynamic topologies and
balancing privacy and accuracy trade-offs. Finally, to supplement and validate
our theoretical results, we run experiments on real-world data from the US
Power Grid Network and electric consumption data from German Households to
estimate the average power consumption of power stations and households under
all privacy regimes and show that our method outperforms existing first-order,
privacy-aware, distributed optimization methods.
