  A Markov decision problem is called reversible if the stationary controlled
Markov chain is reversible under every stationary Markovian strategy. A natural
application in which such problems arise is in the control of
Metropolis-Hastings type dynamics. We characterize all discrete time reversible
Markov decision processes with finite state and actions spaces. We show that
policy iteration algorithm for finding an optimal policy can be significantly
simplified Markov decision problems of this type. We also highlight the
relation between the finite time evolution of the accrual of reward and the
Gaussian free field associated to the controlled Markov chain.
