  Regularising the parameter matrices of neural networks is ubiquitous in
training deep models. Typical regularisation approaches suggest initialising
weights using small random values, and to penalise weights to promote sparsity.
However, these widely used techniques may be less effective in certain
scenarios. Here, we study the Koopman autoencoder model which includes an
encoder, a Koopman operator layer, and a decoder. These models have been
designed and dedicated to tackle physics-related problems with interpretable
dynamics and an ability to incorporate physics-related constraints. However,
the majority of existing work employs standard regularisation practices. In our
work, we take a step toward augmenting Koopman autoencoders with initialisation
and penalty schemes tailored for physics-related settings. Specifically, we
propose the "eigeninit" initialisation scheme that samples initial Koopman
operators from specific eigenvalue distributions. In addition, we suggest the
"eigenloss" penalty scheme that penalises the eigenvalues of the Koopman
operator during training. We demonstrate the utility of these schemes on two
synthetic data sets: a driven pendulum and flow past a cylinder; and two
real-world problems: ocean surface temperatures and cyclone wind fields. We
find on these datasets that eigenloss and eigeninit improves the convergence
rate by up to a factor of 5, and that they reduce the cumulative long-term
prediction error by up to a factor of 3. Such a finding points to the utility
of incorporating similar schemes as an inductive bias in other physics-related
deep learning approaches.
