  In this paper, we study asymptotic properties of problems of control of
stochastic discrete time systems (also known as Markov decision processes) with
time averaging and time discounting optimality criteria, and we establish that
the Ces\`aro and Abel limits of the optimal values in such problems can be
evaluated with the help of a certain infinite-dimensional linear programming
problem and its dual.
