  Federated Learning (FL) has been considered as an appealing framework to
tackle data privacy issues of mobile devices compared to conventional Machine
Learning (ML). Using Edge Servers (ESs) as intermediaries to perform model
aggregation in proximity can reduce the transmission overhead, and it enables
great potentials in low-latency FL, where the hierarchical architecture of FL
(HFL) has been attracted more attention. Designing a proper client selection
policy can significantly improve training performance, and it has been
extensively used in FL studies. However, to the best of our knowledge, there
are no studies focusing on HFL. In addition, client selection for HFL faces
more challenges than conventional FL, e.g., the time-varying connection of
client-ES pairs and the limited budget of the Network Operator (NO). In this
paper, we investigate a client selection problem for HFL, where the NO learns
the number of successful participating clients to improve the training
performance (i.e., select as many clients in each round) as well as under the
limited budget on each ES. An online policy, called Context-aware Online Client
Selection (COCS), is developed based on Contextual Combinatorial Multi-Armed
Bandit (CC-MAB). COCS observes the side-information (context) of local
computing and transmission of client-ES pairs and makes client selection
decisions to maximize NO's utility given a limited budget. Theoretically, COCS
achieves a sublinear regret compared to an Oracle policy on both strongly
convex and non-convex HFL. Simulation results also support the efficiency of
the proposed COCS policy on real-world datasets.
