  We study risk-sensitive multi-agent reinforcement learning under general-sum
Markov games, where agents optimize the entropic risk measure of rewards with
possibly diverse risk preferences. We show that using the regret naively
adapted from existing literature as a performance metric could induce policies
with equilibrium bias that favor the most risk-sensitive agents and overlook
the other agents. To address such deficiency of the naive regret, we propose a
novel notion of regret, which we call risk-balanced regret, and show through a
lower bound that it overcomes the issue of equilibrium bias. Furthermore, we
develop a self-play algorithm for learning Nash, correlated, and coarse
correlated equilibria in risk-sensitive Markov games. We prove that the
proposed algorithm attains near-optimal regret guarantees with respect to the
risk-balanced regret.
