  Federated Learning (FL) endeavors to harness decentralized data while
preserving privacy, facing challenges of performance, scalability, and
collaboration. Asynchronous Federated Learning (AFL) methods have emerged as
promising alternatives to their synchronous counterparts bounded by the slowest
agent, yet they add additional challenges in convergence guarantees, fairness
with respect to compute heterogeneity, and incorporation of staleness in
aggregated updates. Specifically, AFL biases model training heavily towards
agents who can produce updates faster, leaving slower agents behind, who often
also have differently distributed data which is not learned by the global
model. Naively upweighting introduces incentive issues, where true fast
updating agents may falsely report updates at a slower speed to increase their
contribution to model training. We introduce FedStaleWeight, an algorithm
addressing fairness in aggregating asynchronous client updates by employing
average staleness to compute fair re-weightings. FedStaleWeight reframes
asynchronous federated learning aggregation as a mechanism design problem,
devising a weighting strategy that incentivizes truthful compute speed
reporting without favoring faster update-producing agents by upweighting agent
updates based on staleness. Leveraging only observed agent update staleness,
FedStaleWeight results in more equitable aggregation on a per-agent basis. We
both provide theoretical convergence guarantees in the smooth, non-convex
setting and empirically compare FedStaleWeight against the commonly used
asynchronous FedBuff with gradient averaging, demonstrating how it achieves
stronger fairness, expediting convergence to a higher global model accuracy.
Finally, we provide an open-source test bench to facilitate exploration of
buffered AFL aggregation strategies, fostering further research in asynchronous
federated learning paradigms.
