  Maneuver decision-making is the core of unmanned combat aerial vehicle for
autonomous air combat. To solve this problem, we propose an automatic
curriculum reinforcement learning method, which enables agents to learn
effective decisions in air combat from scratch. The range of initial states are
used for distinguishing curricula of different difficulty levels, thereby
maneuver decision is divided into a series of sub-tasks from easy to difficult,
and test results are used to change sub-tasks. As sub-tasks change, agents
gradually learn to complete a series of sub-tasks from easy to difficult,
enabling them to make effective maneuvering decisions to cope with various
states without the need to spend effort designing reward functions. The
ablation studied show that the automatic curriculum learning proposed in this
article is an essential component for training through reinforcement learning,
namely, agents cannot complete effective decisions without curriculum learning.
Simulation experiments show that, after training, agents are able to make
effective decisions given different states, including tracking, attacking and
escaping, which are both rational and interpretable.
