  Federated Learning (FL) has become an active and promising distributed
machine learning paradigm. As a result of statistical heterogeneity, recent
studies clearly show that the performance of popular FL methods (e.g., FedAvg)
deteriorates dramatically due to the client drift caused by local updates. This
paper proposes a novel Federated Learning algorithm (called IGFL), which
leverages both Individual and Group behaviors to mimic distribution, thereby
improving the ability to deal with heterogeneity. Unlike existing FL methods,
our IGFL can be applied to both client and server optimization. As a
by-product, we propose a new attention-based federated learning in the server
optimization of IGFL. To the best of our knowledge, this is the first time to
incorporate attention mechanisms into federated optimization. We conduct
extensive experiments and show that IGFL can significantly improve the
performance of existing federated learning methods. Especially when the
distributions of data among individuals are diverse, IGFL can improve the
classification accuracy by about 13% compared with prior baselines.
