  Machine learning methods are widely used in the natural sciences to model and
predict physical systems from observation data. Yet, they are often used as
poorly understood "black boxes," disregarding existing mathematical structure
and invariants of the problem. Recently, the proposal of Hamiltonian Neural
Networks (HNNs) took a first step towards a unified "gray box" approach, using
physical insight to improve performance for Hamiltonian systems. In this paper,
we explore a significantly improved training method for HNNs, exploiting the
symplectic structure of Hamiltonian systems with a different loss function.
This frees the loss from an artificial lower bound. We mathematically guarantee
the existence of an exact Hamiltonian function which the HNN can learn. This
allows us to prove and numerically analyze the errors made by HNNs which, in
turn, renders them fully explainable. Finally, we present a novel post-training
correction to obtain the true Hamiltonian only from discretized observation
data, up to an arbitrary order.
