  Autonomous Cyber Defence is required to respond to high-tempo cyber-attacks.
To facilitate the research in this challenging area, we explore the utility of
the autonomous cyber operation environments presented as part of the Cyber
Autonomy Gym for Experimentation (CAGE) Challenges, with a specific focus on
CAGE Challenge 2. CAGE Challenge 2 required a defensive Blue agent to defend a
network from an attacking Red agent. We provide a detailed description of the
this challenge and describe the approaches taken by challenge participants.
From the submitted agents, we identify four classes of algorithms, namely,
Single- Agent Deep Reinforcement Learning (DRL), Hierarchical DRL, Ensembles,
and Non-DRL approaches. Of these classes, we found that the hierarchical DRL
approach was the most capable of learning an effective cyber defensive
strategy. Our analysis of the agent policies identified that different
algorithms within the same class produced diverse strategies and that the
strategy used by the defensive Blue agent varied depending on the strategy used
by the offensive Red agent. We conclude that DRL algorithms are a suitable
candidate for autonomous cyber defence applications.
