  Integrating rule-based policies into reinforcement learning promises to
improve data efficiency and generalization in cooperative pursuit problems.
However, most implementations do not properly distinguish the influence of
neighboring robots in observation embedding or inter-robot interaction rules,
leading to information loss and inefficient cooperation. This paper proposes a
cooperative pursuit algorithm named Decentralized Adaptive COOperative Pursuit
via Attention (DACOOP-A) by empowering reinforcement learning with artificial
potential field and attention mechanisms. An attention-based framework is
developed to emphasize important neighbors by concurrently integrating the
learned attention scores into observation embedding and inter-robot interaction
rules. A KL divergence regularization is introduced to alleviate the resultant
learning stability issue. Improvements in data efficiency and generalization
are demonstrated through numerical simulations. Extensive quantitative analysis
and ablation studies are performed to illustrate the advantages of the proposed
modules. Real-world experiments are performed to justify the feasibility of
deploying DACOOP-A in physical systems.
