  We consider Markov decision processes where the state of the chain is only
given at chosen observation times and of a cost. Optimal strategies involve the
optimisation of observation times as well as the subsequent action values. We
consider the finite horizon and discounted infinite horizon problems, as well
as an extension with parameter uncertainty. By including the time elapsed from
observations as part of the augmented Markov system, the value function
satisfies a system of quasi-variational inequalities (QVIs). Such a class of
QVIs can be seen as an extension to the interconnected obstacle problem. We
prove a comparison principle for this class of QVIs, which implies uniqueness
of solutions to our proposed problem. Penalty methods are then utilised to
obtain arbitrarily accurate solutions. Finally, we perform numerical
experiments on three applications which illustrate our framework.
