  Physics-Informed Neural Networks (PINNs) have enabled significant
improvements in modelling physical processes described by partial differential
equations (PDEs). PINNs are based on simple architectures, and learn the
behavior of complex physical systems by optimizing the network parameters to
minimize the residual of the underlying PDE. Current network architectures
share some of the limitations of classical numerical discretization schemes
when applied to non-linear differential equations in continuum mechanics. A
paradigmatic example is the solution of hyperbolic conservation laws that
develop highly localized nonlinear shock waves. Learning solutions of PDEs with
dominant hyperbolic character is a challenge for current PINN approaches, which
rely, like most grid-based numerical schemes, on adding artificial dissipation.
Here, we address the fundamental question of which network architectures are
best suited to learn the complex behavior of non-linear PDEs. We focus on
network architecture rather than on residual regularization. Our new
methodology, called Physics-Informed Attention-based Neural Networks, (PIANNs),
is a combination of recurrent neural networks and attention mechanisms. The
attention mechanism adapts the behavior of the deep neural network to the
non-linear features of the solution, and break the current limitations of
PINNs. We find that PIANNs effectively capture the shock front in a hyperbolic
model problem, and are capable of providing high-quality solutions inside and
beyond the training set.
