  In fitting data with a spline, finding the optimal placement of knots can
significantly improve the quality of the fit. However, the challenging
high-dimensional and non-convex optimization problem associated with completely
free knot placement has been a major roadblock in using this approach. We
present a method that uses particle swarm optimization (PSO) combined with
model selection to address this challenge. The problem of overfitting due to
knot clustering that accompanies free knot placement is mitigated in this
method by explicit regularization, resulting in a significantly improved
performance on highly noisy data. The principal design choices available in the
method are delineated and a statistically rigorous study of their effect on
performance is carried out using simulated data and a wide variety of benchmark
functions. Our results demonstrate that PSO-based free knot placement leads to
a viable and flexible adaptive spline fitting approach that allows the fitting
of both smooth and non-smooth functions.
