  Stochastic optimal control, which has the goal of driving the behavior of
noisy systems, is broadly applicable in science, engineering and artificial
intelligence. Our work introduces Stochastic Optimal Control Matching (SOCM), a
novel Iterative Diffusion Optimization (IDO) technique for stochastic optimal
control that stems from the same philosophy as the conditional score matching
loss for diffusion models. That is, the control is learned via a least squares
problem by trying to fit a matching vector field. The training loss, which is
closely connected to the cross-entropy loss, is optimized with respect to both
the control function and a family of reparameterization matrices which appear
in the matching vector field. The optimization with respect to the
reparameterization matrices aims at minimizing the variance of the matching
vector field. Experimentally, our algorithm achieves lower error than all the
existing IDO techniques for stochastic optimal control for three out of four
control problems, in some cases by an order of magnitude. The key idea
underlying SOCM is the path-wise reparameterization trick, a novel technique
that may be of independent interest. Code at
https://github.com/facebookresearch/SOC-matching
