  Equipping approximate dynamic programming (ADP) with inputconstraints has a
tremendous significance. This enables ADP to be applied tothe systems with
actuator limitations, which is quite common for dynamicalsystems. In a
conventional constrained ADP framework, the optimal control issearched via a
policy iteration algorithm, where the value under a constrainedcontrol is
solved from a Hamilton-Jacobi-Bellman (HJB) equation while theconstrained
control policy is improved based on the current estimated value.This concise
and applicable method has been widely-used. However, the con-vergence of the
existing policy iteration algorithm may possesses a theoreticaldifficulty,
which might be caused by forcibly evaluating the same trajectoryeven though the
control policy has already changed. This problem will beexplored in this paper.
