  Predicting the trajectories of systems with unknown dynamics (\textit{i.e.}
the governing rules) is crucial in various research fields, including physics
and biology. This challenge has gathered significant attention from diverse
communities. Most existing works focus on learning fixed system dynamics within
one single system. However, real-world applications often involve multiple
systems with different types of dynamics or evolving systems with
non-stationary dynamics (dynamics shifts). When data from those systems are
continuously collected and sequentially fed to machine learning models for
training, these models tend to be biased toward the most recently learned
dynamics, leading to catastrophic forgetting of previously observed/learned
system dynamics. To this end, we aim to learn system dynamics via continual
learning. Specifically, we present a novel framework of Mode-switching Graph
ODE (MS-GODE), which can continually learn varying dynamics and encode the
system-specific dynamics into binary masks over the model parameters. During
the inference stage, the model can select the most confident mask based on the
observational data to identify the system and predict future trajectories
accordingly. Empirically, we systematically investigate the task configurations
and compare the proposed MS-GODE with state-of-the-art techniques. More
importantly, we construct a novel benchmark of biological dynamic systems,
featuring diverse systems with disparate dynamics and significantly enriching
the research field of machine learning for dynamic systems.
