  We consider the problem of computing tractable approximations of
time-dependent d x d large positive semi-definite (PSD) matrices defined as
solutions of a matrix differential equation. We propose to use "low-rank plus
diagonal" PSD matrices as approximations that can be stored with a memory cost
being linear in the high dimension d. To constrain the solution of the
differential equation to remain in that subset, we project the derivative at
all times onto the tangent space to the subset, following the methodology of
dynamical low-rank approximation. We derive a closed-form formula for the
projection, and show that after some manipulations it can be computed with a
numerical cost being linear in d, allowing for tractable implementation.
Contrary to previous approaches based on pure low-rank approximations, the
addition of the diagonal term allows for our approximations to be invertible
matrices, that can moreover be inverted with linear cost in d. We apply the
technique to Riccati-like equations, then to two particular problems. Firstly a
low-rank approximation to our recent Wasserstein gradient flow for Gaussian
approximation of posterior distributions in approximate Bayesian inference, and
secondly a novel low-rank approximation of the Kalman filter for
high-dimensional systems. Numerical simulations illustrate the results.
