  We propose ${\tt AdaTS}$, a Thompson sampling algorithm that adapts
sequentially to bandit tasks that it interacts with. The key idea in ${\tt
AdaTS}$ is to adapt to an unknown task prior distribution by maintaining a
distribution over its parameters. When solving a bandit task, that uncertainty
is marginalized out and properly accounted for. ${\tt AdaTS}$ is a
fully-Bayesian algorithm that can be implemented efficiently in several classes
of bandit problems. We derive upper bounds on its Bayes regret that quantify
the loss due to not knowing the task prior, and show that it is small. Our
theory is supported by experiments, where ${\tt AdaTS}$ outperforms prior
algorithms and works well even in challenging real-world problems.
