  In classical Markov Decision Processes (MDPs), action costs and transition
probabilities are assumed to be known, although an accurate estimation of these
parameters is often not possible in practice. This study addresses MDPs under
cost and transition probability uncertainty and aims to provide a mathematical
framework to obtain policies minimizing the risk of high long-term losses due
to not knowing the true system parameters. To this end, we utilize the risk
measure value-at-risk associated with the expected performance of an MDP model
with respect to parameter uncertainty. We provide mixed-integer linear and
nonlinear programming formulations and heuristic algorithms for such
risk-averse models of MDPs under a finite distribution of the uncertain
parameters. Our proposed models and solution methods are illustrated on an
inventory management problem for humanitarian relief operations during a
slow-onset disaster. The results demonstrate the potential of our risk-averse
modeling approach for reducing the risk of highly undesirable outcomes in
uncertain/risky environments.
