  We propose Gauss-Newton's method in function space for the solution of the
Navier-Stokes equations in the physics-informed neural network (PINN)
framework. Upon discretization, this yields a natural gradient method that
provably mimics the function space dynamics. Our computational results
demonstrate close to single-precision accuracy measured in relative $L^2$ norm
on a number of benchmark problems. To the best of our knowledge, this
constitutes the first contribution in the PINN literature that solves the
Navier-Stokes equations to this degree of accuracy. Finally, we show that given
a suitable integral discretization, the proposed optimization algorithm agrees
with Gauss-Newton's method in parameter space. This allows a matrix-free
formulation enabling efficient scalability to large network sizes.
