  We study the problem of allocating impressions to sellers in e-commerce
websites, such as Amazon, eBay or Taobao, aiming to maximize the total revenue
generated by the platform. We employ a general framework of reinforcement
mechanism design, which uses deep reinforcement learning to design efficient
algorithms, taking the strategic behaviour of the sellers into account.
Specifically, we model the impression allocation problem as a Markov decision
process, where the states encode the history of impressions, prices,
transactions and generated revenue and the actions are the possible impression
allocations in each round. To tackle the problem of continuity and
high-dimensionality of states and actions, we adopt the ideas of the DDPG
algorithm to design an actor-critic policy gradient algorithm which takes
advantage of the problem domain in order to achieve convergence and stability.
We evaluate our proposed algorithm, coined IA(GRU), by comparing it against
DDPG, as well as several natural heuristics, under different rationality models
for the sellers - we assume that sellers follow well-known no-regret type
strategies which may vary in their degree of sophistication. We find that
IA(GRU) outperforms all algorithms in terms of the total revenue.
