  Market makers play an important role in providing liquidity to markets by
continuously quoting prices at which they are willing to buy and sell, and
managing inventory risk. In this paper, we build a multi-agent simulation of a
dealer market and demonstrate that it can be used to understand the behavior of
a reinforcement learning (RL) based market maker agent. We use the simulator to
train an RL-based market maker agent with different competitive scenarios,
reward formulations and market price trends (drifts). We show that the
reinforcement learning agent is able to learn about its competitor's pricing
policy; it also learns to manage inventory by smartly selecting asymmetric
prices on the buy and sell sides (skewing), and maintaining a positive (or
negative) inventory depending on whether the market price drift is positive (or
negative). Finally, we propose and test reward formulations for creating risk
averse RL-based market maker agents.
