  Dimensionality reduction on quadratic manifolds augments linear
approximations with quadratic correction terms. Previous works rely on linear
approximations given by projections onto the first few leading principal
components of the training data; however, linear approximations in subspaces
spanned by the leading principal components alone can miss information that are
necessary for the quadratic correction terms to be efficient. In this work, we
propose a greedy method that constructs subspaces from leading as well as later
principal components so that the corresponding linear approximations can be
corrected most efficiently with quadratic terms.
  Properties of the greedily constructed manifolds allow applying linear
algebra reformulations so that the greedy method scales to data points with
millions of dimensions. Numerical experiments demonstrate that an orders of
magnitude higher accuracy is achieved with the greedily constructed quadratic
manifolds compared to manifolds that are based on the leading principal
components alone.
