  In order to drive safely and efficiently under merging scenarios, autonomous
vehicles should be aware of their surroundings and make decisions by
interacting with other road participants. Moreover, different strategies should
be made when the autonomous vehicle is interacting with drivers having
different level of cooperativeness. Whether the vehicle is on the merge-lane or
main-lane will also influence the driving maneuvers since drivers will behave
differently when they have the right-of-way than otherwise. Many traditional
methods have been proposed to solve decision making problems under merging
scenarios. However, these works either are incapable of modeling complicated
interactions or require implementing hand-designed rules which cannot properly
handle the uncertainties in real-world scenarios. In this paper, we proposed an
interaction-aware decision making with adaptive strategies (IDAS) approach that
can let the autonomous vehicle negotiate the road with other drivers by
leveraging their cooperativeness under merging scenarios. A single policy is
learned under the multi-agent reinforcement learning (MARL) setting via the
curriculum learning strategy, which enables the agent to automatically infer
other drivers' various behaviors and make decisions strategically. A masking
mechanism is also proposed to prevent the agent from exploring states that
violate common sense of human judgment and increase the learning efficiency. An
exemplar merging scenario was used to implement and examine the proposed
method.
