  Gradient-based local optimization has been shown to improve results of
genetic programming (GP) for symbolic regression. Several state-of-the-art GP
implementations use iterative nonlinear least squares (NLS) algorithms such as
the Levenberg-Marquardt algorithm for local optimization. The effectiveness of
NLS algorithms depends on appropriate scaling and conditioning of the
optimization problem. This has so far been ignored in symbolic regression and
GP literature. In this study we use a singular value decomposition of NLS
Jacobian matrices to determine the numeric rank and the condition number. We
perform experiments with a GP implementation and six different benchmark
datasets. Our results show that rank-deficient and ill-conditioned Jacobian
matrices occur frequently and for all datasets. The issue is less extreme when
restricting GP tree size and when using many non-linear functions in the
function set.
