  The reliable prediction of the temporal behavior of complex systems is key in
numerous scientific fields. This strong interest is however hindered by
modeling issues: often, the governing equations describing the physics of the
system under consideration are not accessible or, if known, their solution
might require a computational time incompatible with the prediction time
constraints. Not surprisingly, approximating complex systems in a generic
functional format and informing it ex-nihilo from available observations has
become common practice in the age of machine learning, as illustrated by the
numerous successful examples based on deep neural networks. However,
generalizability of the models, margins of guarantee and the impact of data are
often overlooked or examined mainly by relying on prior knowledge of the
physics. We tackle these issues from a different viewpoint, by adopting a
curriculum learning strategy. In curriculum learning, the dataset is structured
such that the training process starts from simple samples towards more complex
ones in order to favor convergence and generalization. The concept has been
developed and successfully applied in robotics and control of systems. Here, we
apply this concept for the learning of complex dynamical systems in a
systematic way. First, leveraging insights from the ergodic theory, we assess
the amount of data sufficient for a-priori guaranteeing a faithful model of the
physical system and thoroughly investigate the impact of the training set and
its structure on the quality of long-term predictions. Based on that, we
consider entropy as a metric of complexity of the dataset; we show how an
informed design of the training set based on the analysis of the entropy
significantly improves the resulting models in terms of generalizability, and
provide insights on the amount and the choice of data required for an effective
data-driven modeling.
