  In this paper, we propose a sampling-based planning and optimal control
method of nonlinear systems under non-differentiable constraints. Motivated by
developing scalable planning algorithms, we consider the optimal motion plan to
be a feedback controller that can be approximated by a weighted sum of given
bases. Given this approximate optimal control formulation, our main
contribution is to introduce importance sampling, specifically, model-reference
adaptive search algorithm, to iteratively compute the optimal weight
parameters, i.e., the weights corresponding to the optimal policy function
approximation given chosen bases. The key idea is to perform the search by
iteratively estimating a parametrized distribution which converges to a Dirac's
Delta that infinitely peaks on the global optimal weights. Then, using this
direct policy search, we incorporated trajectory-based verification to ensure
that, for a class of nonlinear systems, the obtained policy is not only optimal
but robust to bounded disturbances. The correctness and efficiency of the
methods are demonstrated through numerical experiments including linear systems
with a nonlinear cost function and motion planning for a Dubins car.
