  This paper extends to Continuous-Time Jump Markov Decision Processes (CTJMDP)
the classic result for Markov Decision Processes stating that, for a given
initial state distribution, for every policy there is a (randomized) Markov
policy, which can be defined in a natural way, such that at each time instance
the marginal distributions of state-action pairs for these two policies
coincide. It is shown in this paper that this equality takes place for a CTJMDP
if the corresponding Markov policy defines a nonexplosive jump Markov process.
If this Markov process is explosive, then at each time instance the marginal
probability, that a state-action pair belongs to a measurable set of
state-action pairs, is not greater for the described Markov policy than the
same probability for the original policy. These results are used in this paper
to prove that for expected discounted total costs and for average costs per
unit time, for a given initial state distribution, for each policy for a CTJMDP
the described a Markov policy has the same or better performance.
