  Unmanned Combat Aerial Vehicle (UCAV) dogfight, which refers to a fight
between two or more UCAVs usually at close quarters, plays a decisive role on
the aerial battlefields. With the evolution of artificial intelligence,
dogfight progressively transits towards intelligent and autonomous modes.
However, the development of autonomous dogfight policy learning is hindered by
challenges such as weak exploration capabilities, low learning efficiency, and
unrealistic simulated environments. To overcome these challenges, this paper
proposes a novel imitative reinforcement learning framework, which efficiently
leverages expert data while enabling autonomous exploration. The proposed
framework not only enhances learning efficiency through expert imitation, but
also ensures adaptability to dynamic environments via autonomous exploration
with reinforcement learning. Therefore, the proposed framework can learn a
successful dogfight policy of 'pursuit-lock-launch' for UCAVs. To support
data-driven learning, we establish a dogfight environment based on the
Harfang3D sandbox, where we conduct extensive experiments. The results indicate
that the proposed framework excels in multistage dogfight, significantly
outperforms state-of-the-art reinforcement learning and imitation learning
methods. Thanks to the ability of imitating experts and autonomous exploration,
our framework can quickly learn the critical knowledge in complex aerial combat
tasks, achieving up to a 100% success rate and demonstrating excellent
robustness.
