  Given the increase in cybercrime, cybersecurity analysts (i.e. Defenders) are
in high demand. Defenders must monitor an organization's network to evaluate
threats and potential breaches into the network. Adversary simulation is
commonly used to test defenders' performance against known threats to
organizations. However, it is unclear how effective this training process is in
preparing defenders for this highly demanding job. In this paper, we
demonstrate how to use adversarial algorithms to investigate defenders'
learning of defense strategies, using interactive cyber defense games. Our
Interactive Defense Game (IDG) represents a cyber defense scenario that
requires constant monitoring of incoming network alerts and allows a defender
to analyze, remove, and restore services based on the events observed in a
network. The participants in our study faced one of two types of simulated
adversaries. A Beeline adversary is a fast, targeted, and informed attacker;
and a Meander adversary is a slow attacker that wanders the network until it
finds the right target to exploit. Our results suggest that although human
defenders have more difficulty to stop the Beeline adversary initially, they
were able to learn to stop this adversary by taking advantage of their attack
strategy. Participants who played against the Beeline adversary learned to
anticipate the adversary and take more proactive actions, while decreasing
their reactive actions. These findings have implications for understanding how
to help cybersecurity analysts speed up their training.
