  Many real-world auctions are dynamic processes, in which bidders interact and
report information over multiple rounds with the auctioneer. The sequential
decision making aspect paired with imperfect information renders analyzing the
incentive properties of such auctions much more challenging than in the static
case. It is clear that bidders often have incentives for manipulation, but the
full scope of such strategies is not well-understood. We aim to develop a tool
for better understanding the incentive properties in dynamic auctions by using
reinforcement learning to learn the optimal strategic behavior for an auction
participant. We frame the decision problem as a Markov Decision Process, show
its relation to multi-task reinforcement learning and use a soft actor-critic
algorithm with experience relabeling to best-respond against several known
analytical equilibria as well as to find profitable deviations against
exploitable bidder strategies.
