  Many real-world applications can be formulated as multi-agent cooperation
problems, such as network packet routing and coordination of autonomous
vehicles. The emergence of deep reinforcement learning (DRL) provides a
promising approach for multi-agent cooperation through the interaction of the
agents and environments. However, traditional DRL solutions suffer from the
high dimensions of multiple agents with continuous action space during policy
search. Besides, the dynamicity of agents' policies makes the training
non-stationary. To tackle the issues, we propose a hierarchical reinforcement
learning approach with high-level decision-making and low-level individual
control for efficient policy search. In particular, the cooperation of multiple
agents can be learned in high-level discrete action space efficiently. At the
same time, the low-level individual control can be reduced to single-agent
reinforcement learning. In addition to hierarchical reinforcement learning, we
propose an opponent modeling network to model other agents' policies during the
learning process. In contrast to end-to-end DRL approaches, our approach
reduces the learning complexity by decomposing the overall task into sub-tasks
in a hierarchical way. To evaluate the efficiency of our approach, we conduct a
real-world case study in the cooperative lane change scenario. Both simulation
and real-world experiments show the superiority of our approach in the
collision rate and convergence speed.
