  Rocket recycling is a crucial pursuit in aerospace technology, aimed at
reducing costs and environmental impact in space exploration. The primary focus
centers on rocket landing control, involving the guidance of a nonlinear
underactuated rocket with limited fuel in real-time. This challenging task
prompts the application of reinforcement learning (RL), yet goal-oriented
nature of the problem poses difficulties for standard RL algorithms due to the
absence of intermediate reward signals. This paper, for the first time,
significantly elevates the success rate of rocket landing control from 8% with
a baseline controller to 97% on a high-fidelity rocket model using RL. Our
approach, called Random Annealing Jump Start (RAJS), is tailored for real-world
goal-oriented problems by leveraging prior feedback controllers as guide policy
to facilitate environmental exploration and policy learning in RL. In each
episode, the guide policy navigates the environment for the guide horizon,
followed by the exploration policy taking charge to complete remaining steps.
This jump-start strategy prunes exploration space, rendering the problem more
tractable to RL algorithms. The guide horizon is sampled from a uniform
distribution, with its upper bound annealing to zero based on performance
metrics, mitigating distribution shift and mismatch issues in existing methods.
Additional enhancements, including cascading jump start, refined reward and
terminal condition, and action smoothness regulation, further improve policy
performance and practical applicability. The proposed method is validated
through extensive evaluation and Hardware-in-the-Loop testing, affirming the
effectiveness, real-time feasibility, and smoothness of the proposed
controller.
