  Recent works on neural contextual bandits have achieved compelling
performances due to their ability to leverage the strong representation power
of neural networks (NNs) for reward prediction. Many applications of contextual
bandits involve multiple agents who collaborate without sharing raw
observations, thus giving rise to the setting of federated contextual bandits.
Existing works on federated contextual bandits rely on linear or kernelized
bandits, which may fall short when modeling complex real-world reward
functions. So, this paper introduces the federated neural-upper confidence
bound (FN-UCB) algorithm. To better exploit the federated setting, FN-UCB
adopts a weighted combination of two UCBs: $\text{UCB}^{a}$ allows every agent
to additionally use the observations from the other agents to accelerate
exploration (without sharing raw observations), while $\text{UCB}^{b}$ uses an
NN with aggregated parameters for reward prediction in a similar way to
federated averaging for supervised learning. Notably, the weight between the
two UCBs required by our theoretical analysis is amenable to an interesting
interpretation, which emphasizes $\text{UCB}^{a}$ initially for accelerated
exploration and relies more on $\text{UCB}^{b}$ later after enough observations
have been collected to train the NNs for accurate reward prediction (i.e.,
reliable exploitation). We prove sub-linear upper bounds on both the cumulative
regret and the number of communication rounds of FN-UCB, and empirically
demonstrate its competitive performance.
