  Multi-armed bandit problems are receiving a great deal of attention because
they adequately formalize the exploration-exploitation trade-offs arising in
several industrially relevant applications, such as online advertisement and,
more generally, recommendation systems. In many cases, however, these
applications have a strong social component, whose integration in the bandit
algorithm could lead to a dramatic performance increase. For instance, we may
want to serve content to a group of users by taking advantage of an underlying
network of social relationships among them. In this paper, we introduce novel
algorithmic approaches to the solution of such networked bandit problems. More
specifically, we design and analyze a global strategy which allocates a bandit
algorithm to each network node (user) and allows it to "share" signals
(contexts and payoffs) with the neghboring nodes. We then derive two more
scalable variants of this strategy based on different ways of clustering the
graph nodes. We experimentally compare the algorithm and its variants to
state-of-the-art methods for contextual bandits that do not use the relational
information. Our experiments, carried out on synthetic and real-world datasets,
show a marked increase in prediction performance obtained by exploiting the
network structure.
