  The critical threat of phishing emails has been further exacerbated by the
potential of LLMs to generate highly targeted, personalized, and automated
spear phishing attacks. Two critical problems concerning LLM-facilitated
phishing require further investigation: 1) Existing studies on lateral phishing
lack specific examination of LLM integration for large-scale attacks targeting
the entire organization, and 2) Current anti-phishing infrastructure, despite
its extensive development, lacks the capability to prevent LLM-generated
attacks, potentially impacting both employees and IT security incident
management. However, the execution of such investigative studies necessitates a
real-world environment, one that functions during regular business operations
and mirrors the complexity of a large organizational infrastructure. This
setting must also offer the flexibility required to facilitate a diverse array
of experimental conditions, particularly the incorporation of phishing emails
crafted by LLMs. This study is a pioneering exploration into the use of Large
Language Models (LLMs) for the creation of targeted lateral phishing emails,
targeting a large tier 1 university's operation and workforce of approximately
9,000 individuals over an 11-month period. It also evaluates the capability of
email filtering infrastructure to detect such LLM-generated phishing attempts,
providing insights into their effectiveness and identifying potential areas for
improvement. Based on our findings, we propose machine learning-based detection
techniques for such emails to detect LLM-generated phishing emails that were
missed by the existing infrastructure, with an F1-score of 98.96.
