  Approximation of solutions to partial differential equations (PDE) is an
important problem in computational science and engineering. Using neural
networks as an ansatz for the solution has proven a challenge in terms of
training time and approximation accuracy. In this contribution, we discuss how
sampling the hidden weights and biases of the ansatz network from data-agnostic
and data-dependent probability distributions allows us to progress on both
challenges. In most examples, the random sampling schemes outperform iterative,
gradient-based optimization of physics-informed neural networks regarding
training time and accuracy by several orders of magnitude. For time-dependent
PDE, we construct neural basis functions only in the spatial domain and then
solve the associated ordinary differential equation with classical methods from
scientific computing over a long time horizon. This alleviates one of the
greatest challenges for neural PDE solvers because it does not require us to
parameterize the solution in time. For second-order elliptic PDE in Barron
spaces, we prove the existence of sampled networks with $L^2$ convergence to
the solution. We demonstrate our approach on several time-dependent and static
PDEs. We also illustrate how sampled networks can effectively solve inverse
problems in this setting. Benefits compared to common numerical schemes include
spectral convergence and mesh-free construction of basis functions.
