  Even though Google Research Football (GRF) was initially benchmarked and
studied as a single-agent environment in its original paper, recent years have
witnessed an increasing focus on its multi-agent nature by researchers
utilizing it as a testbed for Multi-Agent Reinforcement Learning (MARL).
However, the absence of standardized environment settings and unified
evaluation metrics for multi-agent scenarios hampers the consistent
understanding of various studies. Furthermore, the challenging 5-vs-5 and
11-vs-11 full-game scenarios have received limited thorough examination due to
their substantial training complexities. To address these gaps, this paper
extends the original environment by not only standardizing the environment
settings and benchmarking cooperative learning algorithms across different
scenarios, including the most challenging full-game scenarios, but also by
discussing approaches to enhance football AI from diverse perspectives and
introducing related research tools. Specifically, we provide a distributed and
asynchronous population-based self-play framework with diverse pre-trained
policies for faster training, two football-specific analytical tools for deeper
investigation, and an online leaderboard for broader evaluation. The overall
expectation of this work is to advance the study of Multi-Agent Reinforcement
Learning on Google Research Football environment, with the ultimate goal of
benefiting real-world sports beyond virtual games.
