  In federated learning (FL), clients usually have diverse participation
statistics that are unknown a priori, which can significantly harm the
performance of FL if not handled properly. Existing works aiming at addressing
this problem are usually based on global variance reduction, which requires a
substantial amount of additional memory in a multiplicative factor equal to the
total number of clients. An important open problem is to find a lightweight
method for FL in the presence of clients with unknown participation rates. In
this paper, we address this problem by adapting the aggregation weights in
federated averaging (FedAvg) based on the participation history of each client.
We first show that, with heterogeneous participation statistics, FedAvg with
non-optimal aggregation weights can diverge from the optimal solution of the
original FL objective, indicating the need of finding optimal aggregation
weights. However, it is difficult to compute the optimal weights when the
participation statistics are unknown. To address this problem, we present a new
algorithm called FedAU, which improves FedAvg by adaptively weighting the
client updates based on online estimates of the optimal weights without knowing
the statistics of client participation. We provide a theoretical convergence
analysis of FedAU using a novel methodology to connect the estimation error and
convergence. Our theoretical results reveal important and interesting insights,
while showing that FedAU converges to an optimal solution of the original
objective and has desirable properties such as linear speedup. Our experimental
results also verify the advantage of FedAU over baseline methods with various
participation patterns.
