  Optimal control of diffusion processes is intimately connected to the problem
of solving certain Hamilton-Jacobi-Bellman equations. Building on recent
machine learning inspired approaches towards high-dimensional PDEs, we
investigate the potential of $\textit{iterative diffusion optimisation}$
techniques, in particular considering applications in importance sampling and
rare event simulation, and focusing on problems without diffusion control, with
linearly controlled drift and running costs that depend quadratically on the
control. More generally, our methods apply to nonlinear parabolic PDEs with a
certain shift invariance. The choice of an appropriate loss function being a
central element in the algorithmic design, we develop a principled framework
based on divergences between path measures, encompassing various existing
methods. Motivated by connections to forward-backward SDEs, we propose and
study the novel $\textit{log-variance}$ divergence, showing favourable
properties of corresponding Monte Carlo estimators. The promise of the
developed approach is exemplified by a range of high-dimensional and metastable
numerical examples.
