  In swarm robotics, confrontation including the pursuit-evasion game is a key
scenario. High uncertainty caused by unknown opponents' strategies and dynamic
obstacles complicates the action space into a hybrid decision process. Although
the deep reinforcement learning method is significant for swarm confrontation
since it can handle various sizes, as an end-to-end implementation, it cannot
deal with the hybrid process. Here, we propose a novel hierarchical
reinforcement learning approach consisting of a target allocation layer, a path
planning layer, and the underlying dynamic interaction mechanism between the
two layers, which indicates the quantified uncertainty. It decouples the hybrid
process into discrete allocation and continuous planning layers, with a
probabilistic ensemble model to quantify the uncertainty and regulate the
interaction frequency adaptively. Furthermore, to overcome the unstable
training process introduced by the two layers, we design an integration
training method including pre-training and cross-training, which enhances the
training efficiency and stability. Experiment results in both comparison and
ablation studies validate the effectiveness and generalization performance of
our proposed approach.
