  In this paper, we study a mean-variance optimization problem in an infinite
horizon discrete time discounted Markov decision process (MDP). The objective
is to minimize the variance of system rewards with the constraint of mean
performance. Different from most of works in the literature which require the
mean performance already achieve optimum, we can let the mean discounted
performance equal any constant. The difficulty of this problem is caused by the
quadratic form of the variance function which makes the variance minimization
problem not a standard MDP. By proving the decomposable structure of the
feasible policy space, we transform this constrained variance minimization
problem to an equivalent unconstrained MDP under a new discounted criterion and
a new reward function. The difference of the variances of Markov chains under
any two feasible policies is quantified by a difference formula. Based on the
variance difference formula, a policy iteration algorithm is developed to find
the optimal policy. We also prove the optimality of deterministic policy over
the randomized policy generated in the mean-constrained policy space. Numerical
experiments demonstrate the effectiveness of our approach.
