  Optimization problems on the generalized Stiefel manifold (and products of
it) are prevalent across science and engineering. For example, in computational
science they arise in symmetric (generalized) eigenvalue problems, in nonlinear
eigenvalue problems, and in electronic structures computations, to name a few
problems. In statistics and machine learning, they arise, for example, in
various dimensionality reduction techniques such as canonical correlation
analysis. In deep learning, regularization and improved stability can be
obtained by constraining some layers to have parameter matrices that belong to
the Stiefel manifold. Solving problems on the generalized Stiefel manifold can
be approached via the tools of Riemannian optimization. However, using the
standard geometric components for the generalized Stiefel manifold has two
possible shortcomings: computing some of the geometric components can be too
expensive and convergence can be rather slow in certain cases. Both
shortcomings can be addressed using a technique called Riemannian
preconditioning, which amounts to using geometric components derived by a
precoditioner that defines a Riemannian metric on the constraint manifold. In
this paper we develop the geometric components required to perform Riemannian
optimization on the generalized Stiefel manifold equipped with a non-standard
metric, and illustrate theoretically and numerically the use of those
components and the effect of Riemannian preconditioning for solving
optimization problems on the generalized Stiefel manifold.
