  Command and control (C2) channels are an essential component of many types of
cyber attacks, as they enable attackers to remotely control their
malware-infected machines and execute harmful actions, such as propagating
malicious code across networks, exfiltrating confidential data, or initiating
distributed denial of service (DDoS) attacks. Identifying these C2 channels is
therefore crucial in helping to mitigate and prevent cyber attacks. However,
identifying C2 channels typically involves a manual process, requiring deep
knowledge and expertise in cyber operations. In this paper, we propose a
reinforcement learning (RL) based approach to automatically emulate C2 attack
campaigns using both the normal (public) and the Tor networks. In addition,
payload size and network firewalls are configured to simulate real-world attack
scenarios. Results on a typical network configuration show that the RL agent
can automatically discover resilient C2 attack paths utilizing both Tor-based
and conventional communication channels, while also bypassing network
firewalls.
