  The development of open benchmarking platforms could greatly accelerate the
adoption of AI agents in retail. This paper presents comprehensive simulations
of customer shopping behaviors for the purpose of benchmarking reinforcement
learning (RL) agents that optimize coupon targeting. The difficulty of this
learning problem is largely driven by the sparsity of customer purchase events.
We trained agents using offline batch data comprising summarized customer
purchase histories to help mitigate this effect. Our experiments revealed that
contextual bandit and deep RL methods that are less prone to over-fitting the
sparse reward distributions significantly outperform static policies. This
study offers a practical framework for simulating AI agents that optimize the
entire retail customer journey. It aims to inspire the further development of
simulation tools for retail AI systems.
