  Integrating LLM and reinforcement learning (RL) agent effectively to achieve
complementary performance is critical in high stake tasks like cybersecurity
operations. In this study, we introduce SecurityBot, a LLM agent mentored by
pre-trained RL agents, to support cybersecurity operations. In particularly,
the LLM agent is supported with a profile module to generated behavior
guidelines, a memory module to accumulate local experiences, a reflection
module to re-evaluate choices, and an action module to reduce action space.
Additionally, it adopts the collaboration mechanism to take suggestions from
pre-trained RL agents, including a cursor for dynamic suggestion taken, an
aggregator for multiple mentors' suggestions ranking and a caller for proactive
suggestion asking. Building on the CybORG experiment framework, our experiences
show that SecurityBot demonstrates significant performance improvement compared
with LLM or RL standalone, achieving the complementary performance in the
cybersecurity games.
