  Computing the Nash equilibrium (NE) for N-player non-zerosum stochastic games
is a formidable challenge. Currently, algorithmic methods in stochastic game
theory are unable to compute NE for stochastic games (SGs) for settings in all
but extreme cases in which the players either play as a team or have
diametrically opposed objectives in a two-player setting. This greatly impedes
the application of the SG framework to numerous problems within economics and
practical systems of interest. In this paper, we provide a method of computing
Nash equilibria in nonzero-sum settings and for populations of players more
than two. In particular, we identify a subset of SGs known as stochastic
potential games (SPGs) for which the (Markov perfect) Nash equilibrium can be
computed tractably and in polynomial time. Unlike SGs for which, in general,
computing the NE is PSPACE-hard, we show that SGs with the potential property
are P-Complete. We further demonstrate that for each SPG there is a dual Markov
decision process whose solution coincides with the MP-NE of the SPG. We lastly
provide algorithms that tractably compute the MP-NE for SGs with more than two
players.
