  In the literature, existence of mean-field equilibria has been established
for discrete-time mean field games under both the discounted cost and the
average cost optimality criteria. In this paper, we provide a value iteration
algorithm to compute mean-field equilibrium for both the discounted cost and
the average cost criteria, whose existence proved previously. We establish that
the value iteration algorithm converges to the fixed point of a mean-field
equilibrium operator. Then, using this fixed point, we construct a mean-field
equilibrium. In our value iteration algorithm, we use $Q$-functions instead of
value functions.
