  Multiagent learning settings are inherently more difficult than single-agent
learning because each agent interacts with other simultaneously learning agents
in a shared environment. An effective approach in multiagent reinforcement
learning is to consider the learning process of agents and influence their
future policies toward desirable behaviors from each agent's perspective.
Importantly, if each agent maximizes its long-term rewards by accounting for
the impact of its behavior on the set of convergence policies, the resulting
multiagent system reaches an active equilibrium. While this new solution
concept is general such that standard solution concepts, such as a Nash
equilibrium, are special cases of active equilibria, it is unclear when an
active equilibrium is a preferred equilibrium over other solution concepts. In
this paper, we analyze active equilibria from a game-theoretic perspective by
closely studying examples where Nash equilibria are known. By directly
comparing active equilibria to Nash equilibria in these examples, we find that
active equilibria find more effective solutions than Nash equilibria,
concluding that an active equilibrium is the desired solution for multiagent
learning settings.
