  In recent years, a significant amount of attention has been paid to solve
partial differential equations (PDEs) by deep learning. For example, deep
Galerkin method (DGM) uses the PDE residual in the least-squares sense as the
loss function and a deep neural network (DNN) to approximate the PDE solution.
In this work, we propose a deep mixed residual method (MIM) to solve PDEs with
high-order derivatives. In MIM, we first rewrite a high-order PDE into a
first-order system, very much in the same spirit as local discontinuous
Galerkin method and mixed finite element method in classical numerical methods
for PDEs. We then use the residual of first-order system in the least-squares
sense as the loss function, which is in close connection with least-squares
finite element method. For aforementioned classical numerical methods, the
choice of trail and test functions is important for stability and accuracy
issues in many cases. MIM shares this property when DNNs are employed to
approximate unknowns functions in the first-order system. In one case, we use
nearly the same DNN to approximate all unknown functions and in the other case,
we use totally different DNNs for different unknown functions. In most cases,
MIM provides better approximations (not only for high-derivatives of the PDE
solution but also for the PDE solution itself) than DGM with nearly the same
DNN and the same execution time, sometimes by more than one order of magnitude.
When different DNNs are used, in many cases, MIM provides even better
approximations than MIM with only one DNN, sometimes by more than one order of
magnitude. Therefore, we expect MIM to open up a possibly systematic way to
understand and improve deep learning for solving PDEs from the perspective of
classical numerical analysis.
