  CVaR (Conditional Value at Risk) is a risk metric widely used in finance.
However, dynamically optimizing CVaR is difficult since it is not a standard
Markov decision process (MDP) and the principle of dynamic programming fails.
In this paper, we study the infinite-horizon discrete-time MDP with a long-run
CVaR criterion, from the view of sensitivity-based optimization. By introducing
a pseudo CVaR metric, we derive a CVaR difference formula which quantifies the
difference of long-run CVaR under any two policies. The optimality of
deterministic policies is derived. We obtain a so-called Bellman local
optimality equation for CVaR, which is a necessary and sufficient condition for
local optimal policies and only necessary for global optimal policies. A CVaR
derivative formula is also derived for providing more sensitivity information.
Then we develop a policy iteration type algorithm to efficiently optimize CVaR,
which is shown to converge to local optima in the mixed policy space. We
further discuss some extensions including the mean-CVaR optimization and the
maximization of CVaR. Finally, we conduct numerical experiments relating to
portfolio management to demonstrate the main results. Our work may shed light
on dynamically optimizing CVaR from a sensitivity viewpoint.
