  In this work, we investigate the optimal control problem for continuous-time
Markov decision processes with the random impact of the environment. We provide
conditions to show the existence of optimal controls under finite-horizon
criteria. Under appropriate conditions, the value function is continuous and
satisfies the dynamic programming principle. These results are established by
introducing some restriction on the regularity of the optimal controls and by
developing a new compactification method for continuous-time Markov decision
processes, which is originally used to solve the optimal control problem for
jump-diffusion processes.
