  As a main step in the numerical solution of control problems in continuous
time, the controlled process is approximated by sequences of controlled Markov
chains, thus discretising time and space. A new feature in this context is to
allow for delay in the dynamics. The existence of an optimal strategy with
respect to the cost functional can be guaranteed in the class of relaxed
controls. Weak convergence of the approximating extended Markov chains to the
original process together with convergence of the associated optimal strategies
is established.
