  This paper studies value iteration for infinite horizon contracting Markov
decision processes under convexity assumptions and when the state space is
uncountable. The original value iteration is replaced with a more tractable
form and the fixed points from the modified Bellman operators will be shown to
converge uniformly on compacts sets to their original counterparts. This holds
under various sampling approaches for the random disturbances. Moreover, this
paper will present conditions in which these fixed points form monotone
sequences of lower bounding or upper bounding functions for the original fixed
point. This approach is then demonstrated numerically on a perpetual Bermudan
put option.
