  For a Markov decision process with countably infinite states, the optimal
value may not be achievable in the set of stationary policies. In this paper,
we study the existence conditions of an optimal stationary policy in a
countable-state Markov decision process under the long-run average criterion.
With a properly defined metric on the policy space of ergodic MDPs, the
existence of an optimal stationary policy can be guaranteed by the compactness
of the space and the continuity of the long-run average cost with respect to
the metric. We further extend this condition by some assumptions which can be
easily verified in control problems of specific systems, such as queueing
systems. Our results make a complementary contribution to the literature in the
sense that our method is capable to handle the cost function unbounded from
both below and above, only at the condition of continuity and ergodicity.
Several examples are provided to illustrate the application of our main
results.
