  A significant roadblock to the development of principled multi-agent
reinforcement learning is the fact that desired solution concepts like Nash
equilibria may be intractable to compute. To overcome this obstacle, we take
inspiration from behavioral economics and show that -- by imbuing agents with
important features of human decision-making like risk aversion and bounded
rationality -- a class of risk-averse quantal response equilibria (RQE) become
tractable to compute in all $n$-player matrix and finite-horizon Markov games.
In particular, we show that they emerge as the endpoint of no-regret learning
in suitably adjusted versions of the games. Crucially, the class of
computationally tractable RQE is independent of the underlying game structure
and only depends on agents' degree of risk-aversion and bounded rationality. To
validate the richness of this class of solution concepts we show that it
captures peoples' patterns of play in a number of 2-player matrix games
previously studied in experimental economics. Furthermore, we give a first
analysis of the sample complexity of computing these equilibria in
finite-horizon Markov games when one has access to a generative model and
validate our findings on a simple multi-agent reinforcement learning benchmark.
