  Interval Markov decision processes are a class of Markov models where the
transition probabilities between the states belong to intervals. In this paper,
we study the problem of efficient estimation of the optimal policies in
Interval Markov Decision Processes (IMDPs) with continuous action-space. Given
an IMDP, we show that the pessimistic (resp. the optimistic) value iterations,
i.e., the value iterations under the assumption of a competitive adversary
(resp. cooperative agent), are monotone dynamical systems and are contracting
with respect to the $\ell_{\infty}$-norm. Inspired by this dynamical system
viewpoint, we introduce another IMDP, called the action-space relaxation IMDP.
We show that the action-space relaxation IMDP has two key features: (i) its
optimal value is an upper bound for the optimal value of the original IMDP, and
(ii) its value iterations can be efficiently solved using tools and techniques
from convex optimization. We then consider the policy optimization problems at
each step of the value iterations as a feedback controller of the value
function. Using this system-theoretic perspective, we propose an
iteration-distributed implementation of the value iterations for approximating
the optimal value of the action-space relaxation IMDP.
