  Algorithmic pricing on online e-commerce platforms raises the concern of
tacit collusion, where reinforcement learning algorithms learn to set collusive
prices in a decentralized manner and through nothing more than profit feedback.
This raises the question as to whether collusive pricing can be prevented
through the design of suitable "buy boxes," i.e., through the design of the
rules that govern the elements of e-commerce sites that promote particular
products and prices to consumers. In this paper, we demonstrate that
reinforcement learning (RL) can also be used by platforms to learn buy box
rules that are effective in preventing collusion by RL sellers. For this, we
adopt the methodology of Stackelberg POMDPs, and demonstrate success in
learning robust rules that continue to provide high consumer welfare together
with sellers employing different behavior models or having out-of-distribution
costs for goods.
