  Reinforcement learning is an effective way to solve the decision-making
problems. It is a meaningful and valuable direction to investigate autonomous
air combat maneuver decision-making method based on reinforcement learning.
However, when using reinforcement learning to solve the decision-making
problems with sparse rewards, such as air combat maneuver decision-making, it
costs too much time for training and the performance of the trained agent may
not be satisfactory. In order to solve these problems, the method based on
curriculum learning is proposed. First, three curricula of air combat maneuver
decision-making are designed: angle curriculum, distance curriculum and hybrid
curriculum. These courses are used to train air combat agents respectively, and
compared with the original method without any curriculum. The training results
show that angle curriculum can increase the speed and stability of training,
and improve the performance of the agent; distance curriculum can increase the
speed and stability of agent training; hybrid curriculum has a negative impact
on training, because it makes the agent get stuck at local optimum. The
simulation results show that after training, the agent can handle the
situations where targets come from different directions, and the maneuver
decision results are consistent with the characteristics of missile.
