  Building a single generalist agent with strong zero-shot capability has
recently sparked significant advancements. However, extending this capability
to multi-agent decision making scenarios presents challenges. Most current
works struggle with zero-shot transfer, due to two challenges particular to the
multi-agent settings: (a) a mismatch between centralized training and
decentralized execution; and (b) difficulties in creating generalizable
representations across diverse tasks due to varying agent numbers and action
spaces. To overcome these challenges, we propose a Mask-Based collaborative
learning framework for Multi-Agent decision making (MaskMA). Firstly, we
propose to randomly mask part of the units and collaboratively learn the
policies of unmasked units to handle the mismatch. In addition, MaskMA
integrates a generalizable action representation by dividing the action space
into intrinsic actions solely related to the unit itself and interactive
actions involving interactions with other units. This flexibility allows MaskMA
to tackle tasks with varying agent numbers and thus different action spaces.
Extensive experiments in SMAC reveal MaskMA, with a single model trained on 11
training maps, can achieve an impressive 77.8% average zero-shot win rate on 60
unseen test maps by decentralized execution, while also performing effectively
on other types of downstream tasks (e.g., varied policies collaboration, ally
malfunction, and ad hoc team play).
