  We study mean field games and corresponding $N$-player games in continuous
time over a finite time horizon where the position of each agent belongs to a
finite state space. As opposed to previous works on finite state mean field
games, we use a probabilistic representation of the system dynamics in terms of
stochastic differential equations driven by Poisson random measures. Under mild
assumptions, we prove existence of solutions to the mean field game in relaxed
open-loop as well as relaxed feedback controls. Relying on the probabilistic
representation and a coupling argument, we show that mean field game solutions
provide symmetric $\epsilon_N$-Nash equilibria for the $N$-player game, both in
open-loop and in feedback strategies (not relaxed), with $\epsilon_N\leq
\frac{\text{constant}}{\sqrt{N}}$. Under stronger assumptions, we also find
solutions of the mean field game in ordinary feedback controls and prove
uniqueness either in case of a small time horizon or under monotonicity.
