  This work proposes a novel neural network architecture, called the
Dynamically Controlled Recurrent Neural Network (DCRNN), specifically designed
to model dynamical systems that are governed by ordinary differential equations
(ODEs). The current state vectors of these types of dynamical systems only
depend on their state-space models, along with the respective inputs and
initial conditions. Long Short-Term Memory (LSTM) networks, which have proven
to be very effective for memory-based tasks, may fail to model physical
processes as they tend to memorize, rather than learn how to capture the
information on the underlying dynamics. The proposed DCRNN includes learnable
skip-connections across previously hidden states, and introduces a
regularization term in the loss function by relying on Lyapunov stability
theory. The regularizer enables the placement of eigenvalues of the transfer
function induced by the DCRNN to desired values, thereby acting as an internal
controller for the hidden state trajectory. The results show that, for
forecasting a chaotic dynamical system, the DCRNN outperforms the LSTM in $100$
out of $100$ randomized experiments by reducing the mean squared error of the
LSTM's forecasting by $80.0\% \pm 3.0\%$.
