  Neural operators, as a powerful approximation to the non-linear operators
between infinite-dimensional function spaces, have proved to be promising in
accelerating the solution of partial differential equations (PDE). However, it
requires a large amount of simulated data, which can be costly to collect. This
can be avoided by learning physics from the physics-constrained loss, which we
refer to it as mean squared residual (MSR) loss constructed by the discretized
PDE. We investigate the physical information in the MSR loss, which we called
long-range entanglements, and identify the challenge that the neural network
requires the capacity to model the long-range entanglements in the spatial
domain of the PDE, whose patterns vary in different PDEs. To tackle the
challenge, we propose LordNet, a tunable and efficient neural network for
modeling various entanglements. Inspired by the traditional solvers, LordNet
models the long-range entanglements with a series of matrix multiplications,
which can be seen as the low-rank approximation to the general fully-connected
layers and extracts the dominant pattern with reduced computational cost. The
experiments on solving Poisson's equation and (2D and 3D) Navier-Stokes
equation demonstrate that the long-range entanglements from the MSR loss can be
well modeled by the LordNet, yielding better accuracy and generalization
ability than other neural networks. The results show that the Lordnet can be
$40\times$ faster than traditional PDE solvers. In addition, LordNet
outperforms other modern neural network architectures in accuracy and
efficiency with the smallest parameter size.
