  The curse-of-dimensionality taxes computational resources heavily with
exponentially increasing computational cost as the dimension increases. This
poses great challenges in solving high-dimensional PDEs, as Richard E. Bellman
first pointed out over 60 years ago. While there has been some recent success
in solving numerically partial differential equations (PDEs) in high
dimensions, such computations are prohibitively expensive, and true scaling of
general nonlinear PDEs to high dimensions has never been achieved. We develop a
new method of scaling up physics-informed neural networks (PINNs) to solve
arbitrary high-dimensional PDEs. The new method, called Stochastic Dimension
Gradient Descent (SDGD), decomposes a gradient of PDEs into pieces
corresponding to different dimensions and randomly samples a subset of these
dimensional pieces in each iteration of training PINNs. We prove theoretically
the convergence and other desired properties of the proposed method. We
demonstrate in various diverse tests that the proposed method can solve many
notoriously hard high-dimensional PDEs, including the Hamilton-Jacobi-Bellman
(HJB) and the Schr\"{o}dinger equations in tens of thousands of dimensions very
fast on a single GPU using the PINNs mesh-free approach. Notably, we solve
nonlinear PDEs with nontrivial, anisotropic, and inseparable solutions in
100,000 effective dimensions in 12 hours on a single GPU using SDGD with PINNs.
Since SDGD is a general training methodology of PINNs, it can be applied to any
current and future variants of PINNs to scale them up for arbitrary
high-dimensional PDEs.
