  This paper empirically studies commonly observed training difficulties of
Physics-Informed Neural Networks (PINNs) on dynamical systems. Our results
indicate that fixed points which are inherent to these systems play a key role
in the optimization of the in PINNs embedded physics loss function. We observe
that the loss landscape exhibits local optima that are shaped by the presence
of fixed points. We find that these local optima contribute to the complexity
of the physics loss optimization which can explain common training difficulties
and resulting nonphysical predictions. Under certain settings, e.g., initial
conditions close to fixed points or long simulations times, we show that those
optima can even become better than that of the desired solution.
