  Closed-loop optimal control design for high-dimensional nonlinear systems has
been a long-standing challenge. Traditional methods, such as solving the
associated Hamilton-Jacobi-Bellman equation, suffer from the curse of
dimensionality. Recent literature proposed a new promising approach based on
supervised learning, by leveraging powerful open-loop optimal control solvers
to generate training data and neural networks as efficient high-dimensional
function approximators to fit the closed-loop optimal control. This approach
successfully handles certain high-dimensional optimal control problems but
still performs poorly on more challenging problems. One of the crucial reasons
for the failure is the so-called distribution mismatch phenomenon brought by
the controlled dynamics. In this paper, we investigate this phenomenon and
propose the initial value problem enhanced sampling method to mitigate this
problem. We theoretically prove that this sampling strategy improves over the
vanilla strategy on the classical linear-quadratic regulator by a factor
proportional to the total time duration. We further numerically demonstrate
that the proposed sampling strategy significantly improves the performance on
tested control problems, including the optimal landing problem of a quadrotor
and the optimal reaching problem of a 7 DoF manipulator.
