  We introduce Mean Field Markov games with $N$ players, in which each
individual in a large population interacts with other randomly selected
players. The states and actions of each player in an interaction together
determine the instantaneous payoff for all involved players. They also
determine the transition probabilities to move to the next state. Each
individual wishes to maximize the total expected discounted payoff over an
infinite horizon. We provide a rigorous derivation of the asymptotic behavior
of this system as the size of the population grows to infinity. Under
indistinguishability per type assumption, we show that under any Markov
strategy, the random process consisting of one specific player and the
remaining population converges weakly to a jump process driven by the solution
of a system of differential equations. We characterize the solutions to the
team and to the game problems at the limit of infinite population and use these
to construct near optimal strategies for the case of a finite, but large,
number of players. We show that the large population asymptotic of the
microscopic model is equivalent to a (macroscopic) mean field stochastic game
in which a local interaction is described by a single player against a
population profile (the mean field limit). We illustrate our model to derive
the equations for a dynamic evolutionary Hawk and Dove game with energy level.
