  This work is concerned with solving neural network-based feedback controllers
efficiently for optimal control problems. We first conduct a comparative study
of two prevalent approaches: offline supervised learning and online direct
policy optimization. Albeit the training part of the supervised learning
approach is relatively easy, the success of the method heavily depends on the
optimal control dataset generated by open-loop optimal control solvers. In
contrast, direct policy optimization turns the optimal control problem into an
optimization problem directly without any requirement of pre-computing, but the
dynamics-related objective can be hard to optimize when the problem is
complicated. Our results underscore the superiority of offline supervised
learning in terms of both optimality and training time. To overcome the main
challenges, dataset and optimization, in the two approaches respectively, we
complement them and propose the Pre-train and Fine-tune strategy as a unified
training paradigm for optimal feedback control, which further improves the
performance and robustness significantly. Our code is accessible at
https://github.com/yzhao98/DeepOptimalControl.
