  Previous work on policy learning for Malaria control has often formulated the
problem as an optimization problem assuming the objective function and the
search space have a specific structure. The problem has been formulated as
multi-armed bandits, contextual bandits and a Markov Decision Process in
isolation. Furthermore, an emphasis is put on developing new algorithms
specific to an instance of Malaria control, while ignoring a plethora of
simpler and general algorithms in the literature. In this work, we formally
study the formulation of Malaria control and present a comprehensive analysis
of several formulations used in the literature. In addition, we implement and
analyze several reinforcement learning algorithms in all formulations and
compare them to black box optimization. In contrast to previous work, our
results show that simple algorithms based on Upper Confidence Bounds are
sufficient for learning good Malaria policies, and tend to outperform their
more advanced counterparts on the malaria OpenAI Gym environment.
