  The empirical success of machine learning models with many more parameters
than measurements has generated an interest in the theory of
overparameterisation, i.e., underdetermined models. This paradigm has recently
been studied in domains such as deep learning, where one is interested in good
(local) minima of complex, nonlinear loss functions. Optimisers, like gradient
descent, perform well and consistently reach good solutions. Similarly,
nonlinear optimisation problems are encountered in the field of system
identification. Examples of such high-dimensional problems are optimisation
tasks ensuing from the reconstruction of model states and parameters of an
assumed known dynamical system from observed time series. In this work, we
identify explicit parallels in the benefits of overparameterisation between
what has been analysed in the deep learning context and system identification.
We test multiple chaotic time series models, analysing the optimisation process
for unknown model states and parameters in batch mode. We find that gradient
descent reaches better solutions if we assume more parameters to be unknown. We
hypothesise that, indeed, overparameterisation leads us towards better minima,
and that more degrees of freedom in the optimisation are beneficial so long as
the system is, in principle, observable.
