  A discrete-time method for solving problems in optimal quantum control is
presented. Controlling the time discretized markovian dynamics of a quantum
system can be reduced to a Markov-decision process. We demonstrate this method
in this with a class of simple one qubit systems, which are also discretized in
space. For the task of state preparation we solve the examples both numerically
and analytically with dynamic programming techniques.
