  A/B testing is critical for modern technological companies to evaluate the
effectiveness of newly developed products against standard baselines. This
paper studies optimal designs that aim to maximize the amount of information
obtained from online experiments to estimate treatment effects accurately. We
propose three optimal allocation strategies in a dynamic setting where
treatments are sequentially assigned over time. These strategies are designed
to minimize the variance of the treatment effect estimator when data follow a
non-Markov decision process or a (time-varying) Markov decision process. We
further develop estimation procedures based on existing off-policy evaluation
(OPE) methods and conduct extensive experiments in various environments to
demonstrate the effectiveness of the proposed methodologies. In theory, we
prove the optimality of the proposed treatment allocation design and establish
upper bounds for the mean squared errors of the resulting treatment effect
estimators.
