  In this paper, we consider a class of stochastic optimal control problems
with risk constraints that are expressed as bounded probabilities of failure
for particular initial states. We present here a martingale approach that
diffuses a risk constraint into a martingale to construct time-consistent
control policies. The martingale stands for the level of risk tolerance over
time. By augmenting the system dynamics with the controlled martingale, the
original risk-constrained problem is transformed into a stochastic target
problem. We extend the incremental Markov Decision Process (iMDP) algorithm to
approximate arbitrarily well an optimal feedback policy of the original problem
by sampling in the augmented state space and computing proper boundary
conditions for the reformulated problem. We show that the algorithm is both
probabilistically sound and asymptotically optimal. The performance of the
proposed algorithm is demonstrated on motion planning and control problems
subject to bounded probability of collision in uncertain cluttered
environments.
