  Mean field games formalize dynamic games with a continuum of players and
explicit interaction where the players can have heterogeneous states. As they
additionally yield approximate equilibria of corresponding $N$-player games,
they are of great interest for socio-economic applications. However, most
techniques used for mean field games rely on assumptions that imply that for
each population distribution there is a unique optimizer of the Hamiltonian.
For finite action spaces, this will only hold for trivial models. Thus, the
techniques used so far are not applicable. We propose a model with finite state
and action space, where the dynamics are given by a time-inhomogeneous Markov
chain that might depend on the current population distribution. We show
existence of stationary mean field equilibria in mixed strategies under mild
assumptions and propose techniques to compute all these equilibria. More
precisely, our results allow -- given that the generators are irreducible -- to
characterize the set of stationary mean field equilibria as the set of all
fixed points of a map completely characterized by the transition rates and
rewards for deterministic strategies. Additionally, we propose several partial
results for the case of non-irreducible generators and we demonstrate the
presented techniques on two examples.
