  Multi-vehicle pursuit (MVP) such as autonomous police vehicles pursuing
suspects is important but very challenging due to its mission and safety
critical nature. While multi-agent reinforcement learning (MARL) algorithms
have been proposed for MVP problem in structured grid-pattern roads, the
existing algorithms use randomly training samples in centralized learning,
which leads to homogeneous agents showing low collaboration performance. For
the more challenging problem of pursuing multiple evading vehicles, these
algorithms typically select a fixed target evading vehicle for pursuing
vehicles without considering dynamic traffic situation, which significantly
reduces pursuing success rate. To address the above problems, this paper
proposes a Progression Cognition Reinforcement Learning with Prioritized
Experience for MVP (PEPCRL-MVP) in urban multi-intersection dynamic traffic
scenes. PEPCRL-MVP uses a prioritization network to assess the transitions in
the global experience replay buffer according to the parameters of each MARL
agent. With the personalized and prioritized experience set selected via the
prioritization network, diversity is introduced to the learning process of
MARL, which can improve collaboration and task related performance.
Furthermore, PEPCRL-MVP employs an attention module to extract critical
features from complex urban traffic environments. These features are used to
develop progression cognition method to adaptively group pursuing vehicles.
Each group efficiently target one evading vehicle in dynamic driving
environments. Extensive experiments conducted with a simulator over
unstructured roads of an urban area show that PEPCRL-MVP is superior to other
state-of-the-art methods. Specifically, PEPCRL-MVP improves pursuing efficiency
by 3.95% over TD3-DMAP and its success rate is 34.78% higher than that of
MADDPG. Codes are open sourced.
