  Predicting chaotic dynamical systems is critical in many scientific fields
such as weather prediction, but challenging due to the characterizing sensitive
dependence on initial conditions. Traditional modeling approaches require
extensive domain knowledge, often leading to a shift towards data-driven
methods using machine learning. However, existing research provides
inconclusive results on which machine learning methods are best suited for
predicting chaotic systems. In this paper, we compare different lightweight and
heavyweight machine learning architectures using extensive existing databases,
as well as a newly introduced one that allows for uncertainty quantification in
the benchmark results. We perform hyperparameter tuning based on computational
cost and introduce a novel error metric, the cumulative maximum error, which
combines several desirable properties of traditional metrics, tailored for
chaotic systems. Our results show that well-tuned simple methods, as well as
untuned baseline methods, often outperform state-of-the-art deep learning
models, but their performance can vary significantly with different
experimental setups. These findings underscore the importance of matching
prediction methods to data characteristics and available computational
resources.
