  Multi-agent reinforcement learning (MARL) achieves encouraging performance in
solving complex tasks. However, the safety of MARL policies is one critical
concern that impedes their real-world applications. Popular multi-agent
benchmarks focus on diverse tasks yet provide limited safety support.
Therefore, this work proposes a safety-constrained multi-agent environment:
MatrixWorld, based on the general pursuit-evasion game. Particularly, a
safety-constrained multi-agent action execution model is proposed for the
software implementation of safe multi-agent environments based on diverse
safety definitions. It (1) extends the vertex conflict among homogeneous /
cooperative agents to heterogeneous / adversarial settings, and (2) proposes
three types of resolutions for each type of conflict, aiming at providing
rational and unbiased feedback for safe MARL. Besides, MatrixWorld is also a
lightweight co-evolution framework for the learning of pursuit tasks, evasion
tasks, or both, where more pursuit-evasion variants can be designed based on
different practical meanings of safety. As a brief survey, we review and
analyze the co-evolution mechanism in the multi-agent setting, which clearly
reveals its relationships with autocurricula, self-play, arms races, and
adversarial learning. Thus, MatrixWorld can also serve as the first environment
for autocurricula research, where ideas can be quickly verified and well
understood.
