  As two popular schools of machine learning, online learning and evolutionary
computations have become two important driving forces behind real-world
decision making engines for applications in biomedicine, economics, and
engineering fields. Although there are prior work that utilizes bandits to
improve evolutionary algorithms' optimization process, it remains a field of
blank on how evolutionary approach can help improve the sequential decision
making tasks of online learning agents such as the multi-armed bandits. In this
work, we propose the Genetic Thompson Sampling, a bandit algorithm that keeps a
population of agents and update them with genetic principles such as elite
selection, crossover and mutations. Empirical results in multi-armed bandit
simulation environments and a practical epidemic control problem suggest that
by incorporating the genetic algorithm into the bandit algorithm, our method
significantly outperforms the baselines in nonstationary settings. Lastly, we
introduce EvoBandit, a web-based interactive visualization to guide the readers
through the entire learning process and perform lightweight evaluations on the
fly. We hope to engage researchers into this growing field of research with
this investigation.
