  We present and mathematically analyze an online adjoint algorithm for the
optimization of partial differential equations (PDEs). Traditional adjoint
algorithms would typically solve a new adjoint PDE at each optimization
iteration, which can be computationally costly. In contrast, an online adjoint
algorithm updates the design variables in continuous-time and thus constantly
makes progress towards minimizing the objective function. The online adjoint
algorithm we consider is similar in spirit to the the pseudo-time-stepping,
one-shot method which has been previously proposed. Motivated by the
application of such methods to engineering problems, we mathematically study
the convergence of the online adjoint algorithm. The online adjoint algorithm
relies upon a time-relaxed adjoint PDE which provides an estimate of the
direction of steepest descent. The algorithm updates this estimate continuously
in time, and it asymptotically converges to the exact direction of steepest
descent as $t \rightarrow \infty$. We rigorously prove that the online adjoint
algorithm converges to a critical point of the objective function for
optimizing the PDE. Under appropriate technical conditions, we also prove a
convergence rate for the algorithm. A crucial step in the convergence proof is
a multi-scale analysis of the coupled system for the forward PDE, adjoint PDE,
and the gradient descent ODE for the design variables.
