  In the past year, numerous companies have incorporated Generative AI (GenAI)
capabilities into new and existing applications, forming interconnected
Generative AI (GenAI) ecosystems consisting of semi/fully autonomous agents
powered by GenAI services. While ongoing research highlighted risks associated
with the GenAI layer of agents (e.g., dialog poisoning, membership inference,
prompt leaking, jailbreaking), a critical question emerges: Can attackers
develop malware to exploit the GenAI component of an agent and launch
cyber-attacks on the entire GenAI ecosystem? This paper introduces Morris II,
the first worm designed to target GenAI ecosystems through the use of
adversarial self-replicating prompts. The study demonstrates that attackers can
insert such prompts into inputs that, when processed by GenAI models, prompt
the model to replicate the input as output (replication), engaging in malicious
activities (payload). Additionally, these inputs compel the agent to deliver
them (propagate) to new agents by exploiting the connectivity within the GenAI
ecosystem. We demonstrate the application of Morris II against GenAIpowered
email assistants in two use cases (spamming and exfiltrating personal data),
under two settings (black-box and white-box accesses), using two types of input
data (text and images). The worm is tested against three different GenAI models
(Gemini Pro, ChatGPT 4.0, and LLaVA), and various factors (e.g., propagation
rate, replication, malicious activity) influencing the performance of the worm
are evaluated.
