  We provide a decision theoretic analysis of bandit experiments under local
asymptotics. Working within the framework of diffusion processes, we define
suitable notions of asymptotic Bayes and minimax risk for these experiments.
For normally distributed rewards, the minimal Bayes risk can be characterized
as the solution to a second-order partial differential equation (PDE). Using a
limit of experiments approach, we show that this PDE characterization also
holds asymptotically under both parametric and non-parametric distributions of
the rewards. The approach further describes the state variables it is
asymptotically sufficient to restrict attention to, and thereby suggests a
practical strategy for dimension reduction. The PDEs characterizing minimal
Bayes risk can be solved efficiently using sparse matrix routines or
Monte-Carlo methods. We derive the optimal Bayes and minimax policies from
their numerical solutions. These optimal policies substantially dominate
existing methods such as Thompson sampling; the risk of the latter is often
twice as high.
