  Humanity faces numerous problems of common-pool resource appropriation. This
class of multi-agent social dilemma includes the problems of ensuring
sustainable use of fresh water, common fisheries, grazing pastures, and
irrigation systems. Abstract models of common-pool resource appropriation based
on non-cooperative game theory predict that self-interested agents will
generally fail to find socially positive equilibria---a phenomenon called the
tragedy of the commons. However, in reality, human societies are sometimes able
to discover and implement stable cooperative solutions. Decades of behavioral
game theory research have sought to uncover aspects of human behavior that make
this possible. Most of that work was based on laboratory experiments where
participants only make a single choice: how much to appropriate. Recognizing
the importance of spatial and temporal resource dynamics, a recent trend has
been toward experiments in more complex real-time video game-like environments.
However, standard methods of non-cooperative game theory can no longer be used
to generate predictions for this case. Here we show that deep reinforcement
learning can be used instead. To that end, we study the emergent behavior of
groups of independently learning agents in a partially observed Markov game
modeling common-pool resource appropriation. Our experiments highlight the
importance of trial-and-error learning in common-pool resource appropriation
and shed light on the relationship between exclusion, sustainability, and
inequality.
