  This work focuses on the conservation of quantities such as Hamiltonians,
mass, and momentum when solution fields of partial differential equations are
approximated with nonlinear parametrizations such as deep networks. The
proposed approach builds on Neural Galerkin schemes that are based on the
Dirac--Frenkel variational principle to train nonlinear parametrizations
sequentially in time. We first show that only adding constraints that aim to
conserve quantities in continuous time can be insufficient because the
nonlinear dependence on the parameters implies that even quantities that are
linear in the solution fields become nonlinear in the parameters and thus are
challenging to discretize in time. Instead, we propose Neural Galerkin schemes
that compute at each time step an explicit embedding onto the manifold of
nonlinearly parametrized solution fields to guarantee conservation of
quantities. The embeddings can be combined with standard explicit and implicit
time integration schemes. Numerical experiments demonstrate that the proposed
approach conserves quantities up to machine precision.
