  Progress in multiagent intelligence research is fundamentally limited by the
number and quality of environments available for study. In recent years,
simulated games have become a dominant research platform within reinforcement
learning, in part due to their accessibility and interpretability. Previous
works have targeted and demonstrated success on arcade, first person shooter
(FPS), real-time strategy (RTS), and massive online battle arena (MOBA) games.
Our work considers massively multiplayer online role-playing games (MMORPGs or
MMOs), which capture several complexities of real-world learning that are not
well modeled by any other game genre. We present Neural MMO, a massively
multiagent game environment inspired by MMOs and discuss our progress on two
more general challenges in multiagent systems engineering for AI research:
distributed infrastructure and game IO. We further demonstrate that standard
policy gradient methods and simple baseline models can learn interesting
emergent exploration and specialization behaviors in this setting.
