  Interval Markov Decision Processes (IMDPs) are finite-state uncertain Markov
models, where the transition probabilities belong to intervals. Recently, there
has been a surge of research on employing IMDPs as abstractions of stochastic
systems for control synthesis. However, due to the absence of algorithms for
synthesis over IMDPs with continuous action-spaces, the action-space is assumed
discrete a-priori, which is a restrictive assumption for many applications.
Motivated by this, we introduce continuous-action IMDPs (caIMDPs), where the
bounds on transition probabilities are functions of the action variables, and
study value iteration for maximizing expected cumulative rewards. Specifically,
we decompose the max-min problem associated to value iteration to
$|\mathcal{Q}|$ max problems, where $|\mathcal{Q}|$ is the number of states of
the caIMDP. Then, exploiting the simple form of these max problems, we identify
cases where value iteration over caIMDPs can be solved efficiently (e.g., with
linear or convex programming). We also gain other interesting insights: e.g.,
in certain cases where the action set $\mathcal{A}$ is a polytope, synthesis
over a discrete-action IMDP, where the actions are the vertices of
$\mathcal{A}$, is sufficient for optimality. We demonstrate our results on a
numerical example. Finally, we include a short discussion on employing caIMDPs
as abstractions for control synthesis.
