  Motivated by uncertain parameters encountered in Markov decision processes
(MDPs), we study the effect of parameter uncertainty on Bellman operator-based
methods. Specifically, we consider a family of MDPs where the cost parameters
are from a given compact set. We then define a Bellman operator acting on an
input set of value functions to produce a new set of value functions as the
output under all possible variations in the cost parameters. Finally we prove
the existence of a fixed point of this set-based Bellman operator by showing
that it is a contractive operator on a complete metric space.
