  Correlated Equilibrium (CE) is a well-established solution concept that
captures coordination among agents and enjoys good algorithmic properties. In
real-world multi-agent systems, in addition to being in an equilibrium, agents'
policies are often expected to meet requirements with respect to safety, and
fairness. Such additional requirements can often be expressed in terms of the
state density which measures the state-visitation frequencies during the course
of a game. However, existing CE notions or CE-finding approaches cannot
explicitly specify a CE with particular properties concerning state density;
they do so implicitly by either modifying reward functions or using value
functions as the selection criteria. The resulting CE may thus not fully fulfil
the state-density requirements. In this paper, we propose Density-Based
Correlated Equilibria (DBCE), a new notion of CE that explicitly takes state
density as selection criterion. Concretely, we instantiate DBCE by specifying
different state-density requirements motivated by real-world applications. To
compute DBCE, we put forward the Density Based Correlated Policy Iteration
algorithm for the underlying control problem. We perform experiments on various
games where results demonstrate the advantage of our CE-finding approach over
existing methods in scenarios with state-density concerns.
