  For a countable-state Markov decision process we introduce an embedding which
produces a finite-state Markov decision process. The finite-state embedded
process has the same optimal cost, and moreover, it has the same dynamics as
the original process when restricting to the approximating set. The embedded
process can be used as an approximation which, being finite, is more convenient
for computation and implementation.
