  Deep learning is a powerful tool for solving nonlinear differential
equations, but usually, only the solution corresponding to the flattest local
minimizer can be found due to the implicit regularization of stochastic
gradient descent. This paper proposes a network-based structure probing
deflation method to make deep learning capable of identifying multiple
solutions that are ubiquitous and important in nonlinear physical models.
First, we introduce deflation operators built with known solutions to make
known solutions no longer local minimizers of the optimization energy
landscape. Second, to facilitate the convergence to the desired local
minimizer, a structure probing technique is proposed to obtain an initial guess
close to the desired local minimizer. Together with neural network structures
carefully designed in this paper, the new regularized optimization can converge
to new solutions efficiently. Due to the mesh-free nature of deep learning, the
proposed method is capable of solving high-dimensional problems on complicated
domains with multiple solutions, while existing methods focus on merely one or
two-dimensional regular domains and are more expensive in operation counts.
Numerical experiments also demonstrate that the proposed method could find more
solutions than exiting methods.
