  The modern ability to collect vast quantities of data poses a challenge for
parameter estimation problems. When posed as a nonlinear least squares problem
fitting a model to data, the cost of each iteration grows linearly with the
amount of data and it can easily become prohibitively expensive to perform many
iterations. Here we develop an approach that projects the data onto a
low-dimensional subspace of the high-dimensional data that preserves the
information in the original data. We provide results from both optimization and
statistical perspectives showing that the information is preserved when the
subspace angles between this projection and the Jacobian of the model at the
current iterate remain small. However, for this approach to reduce
computational complexity, both the projected model and Jacobian must be
computed inexpensively. This is a constraint on the pairs of models and
subspaces for which this approach provides a computational speedup. Here we
consider the exponential fitting problem projected onto the range of
Vandermonde matrix, for which the projected model and Jacobian can be computed
in closed form using a generalized geometric sum formula. We further provide an
inexpensive heuristic that picks this Vandermonde matrix so that the subspace
angles with the Jacobian remain small and use this heuristic to update the
subspace during optimization. Although the asymptotic cost still depends on the
data dimension, the overall cost of this sequence of projected nonlinear least
squares problems is less expensive than the original nonlinear least squares
problem. Applied to the exponential fitting problem, this provides an algorithm
that is not only faster in the limit of large data than the conventional
nonlinear least squares approach, but is also faster than subspace based
approaches such as HSVD.
