  We develop two new sets of stable, rank-adaptive Dynamically Orthogonal
Runge-Kutta (DORK) schemes that capture the high-order curvature of the
nonlinear low-rank manifold. The DORK schemes asymptotically approximate the
truncated singular value decomposition at a greatly reduced cost while
preserving mode continuity using newly derived retractions. We show that
arbitrarily high-order optimal perturbative retractions can be obtained, and we
prove that these new retractions are stable. In addition, we demonstrate that
repeatedly applying retractions yields a gradient-descent algorithm on the
low-rank manifold that converges superlinearly when approximating a low-rank
matrix. When approximating a higher-rank matrix, iterations converge linearly
to the best low-rank approximation. We then develop a rank-adaptive retraction
that is robust to overapproximation. Building off of these retractions, we
derive two rank-adaptive integration schemes that dynamically update the
subspace upon which the system dynamics are projected within each time step:
the stable, optimal Dynamically Orthogonal Runge-Kutta (so-DORK) and
gradient-descent Dynamically Orthogonal Runge-Kutta (gd-DORK) schemes. These
integration schemes are numerically evaluated and compared on an
ill-conditioned matrix differential equation, an advection-diffusion partial
differential equation, and a nonlinear, stochastic reaction-diffusion partial
differential equation. Results show a reduced error accumulation rate with the
new stable, optimal and gradient-descent integrators. In addition, we find that
rank adaptation allows for highly accurate solutions while preserving
computational efficiency.
