  As a popular and easy-to-implement machine learning method for solving
differential equations, the physics-informed neural network (PINN) sometimes
may fail and find poor solutions which bias against the exact ones. In this
paper, we establish a framework of modified equation to explain the failure
phenomenon and characterize the implicit bias of a general residual
minimization (RM) method. We provide a simple way to derive the modified
equation which models the numerical solution obtained by RM methods. Next, we
show the modified solution deviates from the original exact solution. The proof
uses a by-product of this paper, that is, a necessary and sufficient condition
on characterizing the singularity of the coefficients. This equivalent
condition can be extended to other types of equations in the future. Finally,
we prove, as a complete characterization of the implicit bias, that RM method
implicitly biases the numerical solution against the exact solution and towards
a modified solution. In this work, we focus on elliptic equations with
discontinuous coefficients, but our approach can be extended to other types of
equations and our understanding of the implicit bias may shed light on further
development of deep learning based methods for solving equations.
