  Multi-agent reinforcement learning holds the key for solving complex tasks
that demand the coordination of learning agents. However, strong coordination
often leads to expensive exploration over the exponentially large state-action
space. A powerful approach is to decompose team works into roles, which are
ideally assigned to agents with the relevant skills. Training agents to
adaptively choose and play emerging roles in a team thus allows the team to
scale to complex tasks and quickly adapt to changing environments. These
promises, however, have not been fully realised by current role-based
multi-agent reinforcement learning methods as they assume either a pre-defined
role structure or a fixed team size. We propose a framework to learn role
assignment and transfer across team sizes. In particular, we train a role
assignment network for small teams by demonstration and transfer the network to
larger teams, which continue to learn through interaction with the environment.
We demonstrate that re-using the role-based credit assignment structure can
foster the learning process of larger reinforcement learning teams to achieve
tasks requiring different roles. Our proposal outperforms competing techniques
in enriched role-enforcing Prey-Predator games and in new scenarios in the
StarCraft II Micro-Management benchmark.
