  We define the hitting time for a Markov decision process (MDP). We do not use
the hitting time of the Markov process induced by the MDP because the induced
chain may not have a stationary distribution. Even it has a stationary
distribution, the stationary distribution may not coincide with the
(normalized) occupancy measure of the MDP. We observe a relationship between
the MDP and the PageRank. Using this observation, we construct an MP whose
stationary distribution coincides with the normalized occupancy measure of the
MDP and we define the hitting time of the MDP as the hitting time of the
associated MP.
