  Multi-armed bandit problems (MABPs) are a special type of optimal control
problem well suited to model resource allocation under uncertainty in a wide
variety of contexts. Since the first publication of the optimal solution of the
classic MABP by a dynamic index rule, the bandit literature quickly diversified
and emerged as an active research topic. Across this literature, the use of
bandit models to optimally design clinical trials became a typical motivating
application, yet little of the resulting theory has ever been used in the
actual design and analysis of clinical trials. To this end, we review two MABP
decision-theoretic approaches to the optimal allocation of treatments in a
clinical trial: the infinite-horizon Bayesian Bernoulli MABP and the
finite-horizon variant. These models possess distinct theoretical properties
and lead to separate allocation rules in a clinical trial design context. We
evaluate their performance compared to other allocation rules, including fixed
randomization. Our results indicate that bandit approaches offer significant
advantages, in terms of assigning more patients to better treatments, and
severe limitations, in terms of their resulting statistical power. We propose a
novel bandit-based patient allocation rule that overcomes the issue of low
power, thus removing a potential barrier for their use in practice.
