  In this paper, we present a communication-efficient federated learning
framework inspired by quantized compressed sensing. The presented framework
consists of gradient compression for wireless devices and gradient
reconstruction for a parameter server (PS). Our strategy for gradient
compression is to sequentially perform block sparsification, dimensional
reduction, and quantization. Thanks to gradient sparsification and
quantization, our strategy can achieve a higher compression ratio than one-bit
gradient compression. For accurate aggregation of the local gradients from the
compressed signals at the PS, we put forth an approximate minimum mean square
error (MMSE) approach for gradient reconstruction using the
expectation-maximization generalized-approximate-message-passing (EM-GAMP)
algorithm. Assuming Bernoulli Gaussian-mixture prior, this algorithm
iteratively updates the posterior mean and variance of local gradients from the
compressed signals. We also present a low-complexity approach for the gradient
reconstruction. In this approach, we use the Bussgang theorem to aggregate
local gradients from the compressed signals, then compute an approximate MMSE
estimate of the aggregated gradient using the EM-GAMP algorithm. We also
provide a convergence rate analysis of the presented framework. Using the MNIST
dataset, we demonstrate that the presented framework achieves almost identical
performance with the case that performs no compression, while significantly
reducing communication overhead for federated learning.
