  Physics-informed Neural Networks (PINNs) have recently gained popularity due
to their effective approximation of partial differential equations (PDEs) using
deep neural networks (DNNs). However, their out of domain behavior is not well
understood, with previous work speculating that the presence of high frequency
components in the solution function might be to blame for poor extrapolation
performance. In this paper, we study the extrapolation behavior of PINNs on a
representative set of PDEs of different types, including high-dimensional PDEs.
We find that failure to extrapolate is not caused by high frequencies in the
solution function, but rather by shifts in the support of the Fourier spectrum
over time. We term these spectral shifts and quantify them by introducing a
Weighted Wasserstein-Fourier distance (WWF). We show that the WWF can be used
to predict PINN extrapolation performance, and that in the absence of
significant spectral shifts, PINN predictions stay close to the true solution
even in extrapolation. Finally, we propose a transfer learning-based strategy
to mitigate the effects of larger spectral shifts, which decreases
extrapolation errors by up to 82%.
