  Federated learning (FL) is an appealing paradigm for learning a global model
among distributed clients while preserving data privacy. Driven by the demand
for high-quality user experiences, evaluating the well-trained global model
after the FL process is crucial. In this paper, we propose a closed-loop model
analytics framework that allows for effective evaluation of the trained global
model using clients' local data. To address the challenges posed by system and
data heterogeneities in the FL process, we study a goal-directed client
selection problem based on the model analytics framework by selecting a subset
of clients for the model training. This problem is formulated as a stochastic
multi-armed bandit (SMAB) problem. We first put forth a quick initial upper
confidence bound (Quick-Init UCB) algorithm to solve this SMAB problem under
the federated analytics (FA) framework. Then, we further propose a belief
propagation-based UCB (BP-UCB) algorithm under the democratized analytics (DA)
framework. Moreover, we derive two regret upper bounds for the proposed
algorithms, which increase logarithmically over the time horizon. The numerical
results demonstrate that the proposed algorithms achieve nearly optimal
performance, with a gap of less than 1.44% and 3.12% under the FA and DA
frameworks, respectively.
