  The continuous dynamics of natural systems has been effectively modelled
using Neural Ordinary Differential Equations (Neural ODEs). However, for
accurate and meaningful predictions, it is crucial that the models follow the
underlying rules or laws that govern these systems. In this work, we propose a
self-adaptive penalty algorithm for Neural ODEs to enable modelling of
constrained natural systems. The proposed self-adaptive penalty function can
dynamically adjust the penalty parameters. The explicit introduction of prior
knowledge helps to increase the interpretability of Neural ODE -based models.
We validate the proposed approach by modelling three natural systems with prior
knowledge constraints: population growth, chemical reaction evolution, and
damped harmonic oscillator motion. The numerical experiments and a comparison
with other penalty Neural ODE approaches and \emph{vanilla} Neural ODE,
demonstrate the effectiveness of the proposed self-adaptive penalty algorithm
for Neural ODEs in modelling constrained natural systems. Moreover, the
self-adaptive penalty approach provides more accurate and robust models with
reliable and meaningful predictions.
