  In federated learning (FL), the common paradigm that FedAvg proposes and most
algorithms follow is that clients train local models with their private data,
and the model parameters are shared for central aggregation, mostly averaging.
In this paradigm, the communication cost is often a challenge, as modern
massive neural networks can contain millions to billions parameters. We suggest
that clients do not share model parameters but local data summaries, to
decrease the cost of sharing. We develop a new algorithm FedLog with Bayesian
inference, which shares only sufficient statistics of local data. FedLog
transmits messages as small as the last layer of the original model. We
conducted comprehensive experiments to show we outperform other FL algorithms
that aim at decreasing the communication cost. To provide formal privacy
guarantees, we further extend FedLog with differential privacy and show the
trade-off between privacy budget and accuracy.
