  The emergence of complex life on Earth is often attributed to the arms race
that ensued from a huge number of organisms all competing for finite resources.
We present an artificial intelligence research environment, inspired by the
human game genre of MMORPGs (Massively Multiplayer Online Role-Playing Games,
a.k.a. MMOs), that aims to simulate this setting in microcosm. As with MMORPGs
and the real world alike, our environment is persistent and supports a large
and variable number of agents. Our environment is well suited to the study of
large-scale multiagent interaction: it requires that agents learn robust combat
and navigation policies in the presence of large populations attempting to do
the same. Baseline experiments reveal that population size magnifies and
incentivizes the development of skillful behaviors and results in agents that
outcompete agents trained in smaller populations. We further show that the
policies of agents with unshared weights naturally diverge to fill different
niches in order to avoid competition.
