  The discretization of the deep Ritz method [18] for the Poisson equation
leads to a high-dimensional non-convex minimization problem, that is difficult
and expensive to solve numerically. In this paper, we consider the shallow Ritz
approximation to one-dimensional diffusion problems and introduce an effective
and efficient iterative method, a damped block Newton (dBN) method, for solving
the resulting non-convex minimization problem.
  The method employs the block Gauss-Seidel method as an outer iteration by
dividing the parameters of a shallow neural network into the linear parameters
(the weights and bias of the output layer) and the non-linear parameters (the
weights and bias of the hidden layer). Per each outer iteration, the linear and
the non-linear parameters are updated by exact inversion and one step of a
damped Newton method, respectively. Inverses of the coefficient matrix and the
Hessian matrix are tridiagonal and diagonal, respectively, and hence the cost
of each dBN iteration is $\mathcal{O}(n)$. To move the breakpoints (the
non-linear parameters) more efficiently, we propose an adaptive damped block
Newton (AdBN) method by combining the dBN with the adaptive neuron enhancement
(ANE) method [25]. Numerical examples demonstrate the ability of dBN and AdBN
not only to move the breakpoints quickly and efficiently but also to achieve a
nearly optimal order of convergence for AdBN. These iterative solvers are
capable of outperforming BFGS for select examples.
