  The availability of challenging benchmarks has played a key role in the
recent progress of machine learning. In cooperative multi-agent reinforcement
learning, the StarCraft Multi-Agent Challenge (SMAC) has become a popular
testbed for centralised training with decentralised execution. However, after
years of sustained improvement on SMAC, algorithms now achieve near-perfect
performance. In this work, we conduct new analysis demonstrating that SMAC
lacks the stochasticity and partial observability to require complex
*closed-loop* policies. In particular, we show that an *open-loop* policy
conditioned only on the timestep can achieve non-trivial win rates for many
SMAC scenarios. To address this limitation, we introduce SMACv2, a new version
of the benchmark where scenarios are procedurally generated and require agents
to generalise to previously unseen settings (from the same distribution) during
evaluation. We also introduce the extended partial observability challenge
(EPO), which augments SMACv2 to ensure meaningful partial observability. We
show that these changes ensure the benchmark requires the use of *closed-loop*
policies. We evaluate state-of-the-art algorithms on SMACv2 and show that it
presents significant challenges not present in the original benchmark. Our
analysis illustrates that SMACv2 addresses the discovered deficiencies of SMAC
and can help benchmark the next generation of MARL methods. Videos of training
are available at https://sites.google.com/view/smacv2.
