  Recent advancements have significantly improved automated task-solving
capabilities using autonomous agents powered by large language models (LLMs).
However, most LLM-based agents focus on dialogue, programming, or specialized
domains, leaving gaps in addressing generative AI safety tasks. These gaps are
primarily due to the challenges posed by LLM hallucinations and the lack of
clear guidelines. In this paper, we propose Atlas, an advanced LLM-based
multi-agent framework that integrates an efficient fuzzing workflow to target
generative AI models, specifically focusing on jailbreak attacks against
text-to-image (T2I) models with safety filters. Atlas utilizes a
vision-language model (VLM) to assess whether a prompt triggers the T2I model's
safety filter. It then iteratively collaborates with both LLM and VLM to
generate an alternative prompt that bypasses the filter. Atlas also enhances
the reasoning abilities of LLMs in attack scenarios by leveraging multi-agent
communication, in-context learning (ICL) memory mechanisms, and the
chain-of-thought (COT) approach. Our evaluation demonstrates that Atlas
successfully jailbreaks several state-of-the-art T2I models in a black-box
setting, which are equipped with multi-modal safety filters. In addition, Atlas
outperforms existing methods in both query efficiency and the quality of the
generated images.
