  In this paper, we propose a navigation algorithm oriented to multi-agent
environment. This algorithm is expressed as a hierarchical framework that
contains a Hidden Markov Model (HMM) and a Deep Reinforcement Learning (DRL)
structure. For simplification, we term our method Hierarchical Navigation
Reinforcement Network (HNRN). In high- level architecture, we train an HMM to
evaluate the agent's perception to obtain a score. According to this score,
adaptive control action will be chosen. While in low-level architecture, two
sub-systems are introduced, one is a differential target- driven system, which
aims at heading to the target; the other is a collision avoidance DRL system,
which is used for avoiding dynamic obstacles. The advantage of this
hierarchical structure is decoupling the target-driven and collision avoidance
tasks, leading to a faster and more stable model to be trained. The experiments
indicate that our algorithm has higher learning efficiency and rate of success
than traditional Velocity Obstacle (VO) algorithms or hybrid DRL method.
