  Policy iteration is a widely used technique to solve the Hamilton Jacobi
Bellman (HJB) equation, which arises from nonlinear optimal feedback control
theory. Its convergence analysis has attracted much attention in the
unconstrained case. Here we analyze the case with control constraints both for
the HJB equations which arise in deterministic and in stochastic control cases.
The linear equations in each iteration step are solved by an implicit upwind
scheme. Numerical examples are conducted to solve the HJB equation with control
constraints and comparisons are shown with the unconstrained cases.
