  Solving high-dimensional partial differential equations is a recurrent
challenge in economics, science and engineering. In recent years, a great
number of computational approaches have been developed, most of them relying on
a combination of Monte Carlo sampling and deep learning based approximation.
For elliptic and parabolic problems, existing methods can broadly be classified
into those resting on reformulations in terms of $\textit{backward stochastic
differential equations}$ (BSDEs) and those aiming to minimize a regression-type
$L^2$-error ($\textit{physics-informed neural networks}$, PINNs). In this
paper, we review the literature and suggest a methodology based on the novel
$\textit{diffusion loss}$ that interpolates between BSDEs and PINNs. Our
contribution opens the door towards a unified understanding of numerical
approaches for high-dimensional PDEs, as well as for implementations that
combine the strengths of BSDEs and PINNs. The diffusion loss furthermore bears
close similarities to $\textit{(least squares) temporal difference}$ objectives
found in reinforcement learning. We also discuss eigenvalue problems and
perform extensive numerical studies, including calculations of the ground state
for nonlinear Schr\"odinger operators and committor functions relevant in
molecular dynamics.
