  The numerical solution of differential equations using machine learning-based
approaches has gained significant popularity. Neural network-based
discretization has emerged as a powerful tool for solving differential
equations by parameterizing a set of functions. Various approaches, such as the
deep Ritz method and physics-informed neural networks, have been developed for
numerical solutions. Training algorithms, including gradient descent and greedy
algorithms, have been proposed to solve the resulting optimization problems. In
this paper, we focus on the variational formulation of the problem and propose
a Gauss- Newton method for computing the numerical solution. We provide a
comprehensive analysis of the superlinear convergence properties of this
method, along with a discussion on semi-regular zeros of the vanishing
gradient. Numerical examples are presented to demonstrate the efficiency of the
proposed Gauss-Newton method.
