  Maneuvering in dense traffic is a challenging task for autonomous vehicles
because it requires reasoning about the stochastic behaviors of many other
participants. In addition, the agent must achieve the maneuver within a limited
time and distance. In this work, we propose a combination of reinforcement
learning and game theory to learn merging behaviors. We design a training
curriculum for a reinforcement learning agent using the concept of level-$k$
behavior. This approach exposes the agent to a broad variety of behaviors
during training, which promotes learning policies that are robust to model
discrepancies. We show that our approach learns more efficient policies than
traditional training methods.
