  We establish the existence of optimal scheduling strategies for time-bounded
reachability in continuous-time Markov decision processes, and of co-optimal
strategies for continuous-time Markov games. Furthermore, we show that optimal
control does not only exist, but has a surprisingly simple structure: The
optimal schedulers from our proofs are deterministic and timed-positional, and
the bounded time can be divided into a finite number of intervals, in which the
optimal strategies are positional. That is, we demonstrate the existence of
finite optimal control. Finally, we show that these pleasant properties of
Markov decision processes extend to the more general class of continuous-time
Markov games, and that both early and late schedulers show this behaviour.
