  Multi-agent reinforcement learning has recently shown great promise as an
approach to networked system control. Arguably, one of the most difficult and
important tasks for which large scale networked system control is applicable is
common-pool resource management. Crucial common-pool resources include arable
land, fresh water, wetlands, wildlife, fish stock, forests and the atmosphere,
of which proper management is related to some of society's greatest challenges
such as food security, inequality and climate change. Here we take inspiration
from a recent research program investigating the game-theoretic incentives of
humans in social dilemma situations such as the well-known tragedy of the
commons. However, instead of focusing on biologically evolved human-like
agents, our concern is rather to better understand the learning and operating
behaviour of engineered networked systems comprising general-purpose
reinforcement learning agents, subject only to nonbiological constraints such
as memory, computation and communication bandwidth. Harnessing tools from
empirical game-theoretic analysis, we analyse the differences in resulting
solution concepts that stem from employing different information structures in
the design of networked multi-agent systems. These information structures
pertain to the type of information shared between agents as well as the
employed communication protocol and network topology. Our analysis contributes
new insights into the consequences associated with certain design choices and
provides an additional dimension of comparison between systems beyond
efficiency, robustness, scalability and mean control performance.
