  Differential Dynamic Programming (DDP) has become a well established method
for unconstrained trajectory optimization. Despite its several applications in
robotics and controls however, a widely successful constrained version of the
algorithm has yet to be developed. This paper builds upon penalty methods and
active-set approaches, towards designing a Dynamic Programming-based
methodology for constrained optimal control. Regarding the former, our
derivation employs a constrained version of Bellman's principle of optimality,
by introducing a set of auxiliary slack variables in the backward pass. In
parallel, we show how Augmented Lagrangian methods can be naturally
incorporated within DDP, by utilizing a particular set of penalty-Lagrangian
functions that preserve second-order differentiability. We demonstrate
experimentally that our extensions (individually and combinations thereof)
enhance significantly the convergence properties of the algorithm, and
outperform previous approaches on a large number of simulated scenarios.
