  Statistical heterogeneity of data present at client devices in a federated
learning (FL) system renders the training of a global model in such systems
difficult. Particularly challenging are the settings where due to resource
constraints only a small fraction of clients can participate in any given round
of FL. Recent approaches to training a global model in FL systems with non-IID
data have focused on developing client selection methods that aim to sample
clients with more informative updates of the model. However, existing client
selection techniques either introduce significant computation overhead or
perform well only in the scenarios where clients have data with similar
heterogeneity profiles. In this paper, we propose HiCS-FL (Federated Learning
via Hierarchical Clustered Sampling), a novel client selection method in which
the server estimates statistical heterogeneity of a client's data using the
client's update of the network's output layer and relies on this information to
cluster and sample the clients. We analyze the ability of the proposed
techniques to compare heterogeneity of different datasets, and characterize
convergence of the training process that deploys the introduced client
selection method. Extensive experimental results demonstrate that in non-IID
settings HiCS-FL achieves faster convergence and lower training variance than
state-of-the-art FL client selection schemes. Notably, HiCS-FL drastically
reduces computation cost compared to existing selection schemes and is
adaptable to different heterogeneity scenarios.
