  In distributed optimization and learning, several machines alternate between
local computations in parallel and communication with a distant server.
Communication is usually slow and costly and forms the main bottleneck. This is
particularly true in federated learning, where a large number of users
collaborate toward a global training task. In addition, it is desirable for a
robust algorithm to allow for partial participation, since it is often the case
that some clients are not able to participate to the entire process and are
idle at certain times. Two strategies are popular to reduce the communication
burden: 1) local training, which consists in communicating less frequently, or
equivalently performing more local computations between the communication
rounds; and 2) compression, whereby compressed information instead of
full-dimensional vectors is communicated. We propose TAMUNA, the first
algorithm for distributed optimization that leveraged the two strategies of
local training and compression jointly and allows for partial participation. In
the strongly convex setting, TAMUNA converges linearly to the exact solution
and provably benefits from the two mechanisms: it exhibits a doubly-accelerated
convergence rate, with respect to the condition number of the functions and the
model dimension.
