  Multi-agent reinforcement learning (MARL) methods, while effective in
zero-sum or positive-sum games, often yield suboptimal outcomes in general-sum
games where cooperation is essential for achieving globally optimal outcomes.
Matrix game social dilemmas, which abstract key aspects of general-sum
interactions, such as cooperation, risk, and trust, fail to model the temporal
and spatial dynamics characteristic of real-world scenarios. In response, our
study extends matrix game social dilemmas into more complex, higher-dimensional
MARL environments. We adapt a gridworld implementation of the Stag Hunt dilemma
to more closely match the decision-space of a one-shot matrix game while also
introducing variable environment complexity. Our findings indicate that as
complexity increases, MARL agents trained in these environments converge to
suboptimal strategies, consistent with the risk-dominant Nash equilibria
strategies found in matrix games. Our work highlights the impact of environment
complexity on achieving optimal outcomes in higher-dimensional game-theoretic
MARL environments.
