  How to explore efficiently is a central problem in multi-armed bandits. In
this paper, we introduce the metadata-based multi-task bandit problem, where
the agent needs to solve a large number of related multi-armed bandit tasks and
can leverage some task-specific features (i.e., metadata) to share knowledge
across tasks. As a general framework, we propose to capture task relations
through the lens of Bayesian hierarchical models, upon which a Thompson
sampling algorithm is designed to efficiently learn task relations, share
information, and minimize the cumulative regrets. Two concrete examples for
Gaussian bandits and Bernoulli bandits are carefully analyzed. The Bayes regret
for Gaussian bandits clearly demonstrates the benefits of information sharing
with our algorithm. The proposed method is further supported by extensive
experiments.
