  Bandit algorithms are often used in the e-commerce industry to train Machine
Learning (ML) systems when pre-labeled data is unavailable. However, the
industry setting poses various challenges that make implementing bandit
algorithms in practice non-trivial. In this paper, we elaborate on the
challenges of off-policy optimisation, delayed reward, concept drift, reward
design, and business rules constraints that practitioners at Booking.com
encounter when applying bandit algorithms. Our main contributions is an
extension to the Open Bandit Pipeline (OBP) framework. We provide simulation
components for some of the above-mentioned challenges to provide future
practitioners, researchers, and educators with a resource to address challenges
encountered in the e-commerce industry.
