  Role-based learning is a promising approach to improving the performance of
Multi-Agent Reinforcement Learning (MARL). Nevertheless, without manual
assistance, current role-based methods cannot guarantee stably discovering a
set of roles to effectively decompose a complex task, as they assume either a
predefined role structure or practical experience for selecting
hyperparameters. In this article, we propose a mathematical Structural
Information principles-based Role Discovery method, namely SIRD, and then
present a SIRD optimizing MARL framework, namely SR-MARL, for multi-agent
collaboration. The SIRD transforms role discovery into a hierarchical action
space clustering. Specifically, the SIRD consists of structuralization,
sparsification, and optimization modules, where an optimal encoding tree is
generated to perform abstracting to discover roles. The SIRD is agnostic to
specific MARL algorithms and flexibly integrated with various value function
factorization approaches. Empirical evaluations on the StarCraft II
micromanagement benchmark demonstrate that, compared with state-of-the-art MARL
algorithms, the SR-MARL framework improves the average test win rate by 0.17%,
6.08%, and 3.24%, and reduces the deviation by 16.67%, 30.80%, and 66.30%,
under easy, hard, and super hard scenarios.
