  The goal of this paper is to investigate new and simple convergence analysis
of dynamic programming for linear quadratic regulator problem of discrete-time
linear time-invariant systems. In particular, bounds on errors are given in
terms of both matrix inequalities and matrix norm. Under a mild assumption on
the initial parameter, we prove that the Q-value iteration exponentially
converges to the optimal solution. Moreover, a global asymptotic convergence is
also presented. These results are then extended to the policy iteration. We
prove that in contrast to the Q-value iteration, the policy iteration always
converges exponentially fast. An example is given to illustrate the results.
