  The dynamics in games involving multiple players, who adaptively learn from
their past experience, is not yet well understood. We analyzed a class of
stochastic games with Markov strategies in which players choose their actions
probabilistically. This class is formulated as a $k^{\text{th}}$ order Markov
process, in which the probability of choice is a function of $k$ past states.
With a reasonably large $k$ or with the limit $k \to \infty$, numerical
analysis of this random process is unfeasible. This study developed a technique
which gives the marginal probability of the stationary distribution of the
infinite-order Markov process, which can be constructed recursively. We applied
this technique to analyze an iterated prisoner's dilemma game with two players
who learn using infinite memory.
