  Optimization for robot control tasks, spanning various methodologies,
includes Model Predictive Control (MPC). However, the complexity of the system,
such as non-convex and non-differentiable cost functions and prolonged planning
horizons often drastically increases the computation time, limiting MPC's
real-world applicability. Prior works in speeding up the optimization have
limitations on solving convex problem and generalizing to hold out domains. To
overcome this challenge, we develop a novel framework aiming at expediting
optimization processes. In our framework, we combine offline self-supervised
learning and online fine-tuning through reinforcement learning to improve the
control performance and reduce optimization time. We demonstrate the
effectiveness of our method on a novel, challenging Formula-1-track driving
task, achieving 3.9\% higher performance in optimization time and 3.6\% higher
performance in tracking accuracy on challenging holdout tracks.
