  Robot design optimization, imitation learning and system identification share
a common problem which requires optimization over robot or task parameters at
the same time as optimizing the robot motion. To solve these problems, we can
use differentiable optimal control for which the gradients of the robot's
motion with respect to the parameters are required. We propose a method to
efficiently compute these gradients analytically via the differential dynamic
programming (DDP) algorithm using sensitivity analysis (SA). We show that we
must include second-order dynamics terms when computing the gradients. However,
we do not need to include them when computing the motion. We validate our
approach on the pendulum and double pendulum systems. Furthermore, we compare
against using the derivatives of the iterative linear quadratic regulator
(iLQR), which ignores these second-order terms everywhere, on a co-design task
for the Kinova arm, where we optimize the link lengths of the robot for a
target reaching task. We show that optimizing using iLQR gradients diverges as
ignoring the second-order dynamics affects the computation of the derivatives.
Instead, optimizing using DDP gradients converges to the same optimum for a
range of initial designs allowing our formulation to scale to complex systems.
