  Despite the widespread application of large language models (LLMs) across
various tasks, recent studies indicate that they are susceptible to jailbreak
attacks, which can render their defense mechanisms ineffective. However,
previous jailbreak research has frequently been constrained by limited
universality, suboptimal efficiency, and a reliance on manual crafting. In
response, we rethink the approach to jailbreaking LLMs and formally define
three essential properties from the attacker' s perspective, which contributes
to guiding the design of jailbreak methods. We further introduce AutoBreach, a
novel method for jailbreaking LLMs that requires only black-box access.
Inspired by the versatility of wordplay, AutoBreach employs a wordplay-guided
mapping rule sampling strategy to generate a variety of universal mapping rules
for creating adversarial prompts. This generation process leverages LLMs'
automatic summarization and reasoning capabilities, thus alleviating the manual
burden. To boost jailbreak success rates, we further suggest sentence
compression and chain-of-thought-based mapping rules to correct errors and
wordplay misinterpretations in target LLMs. Additionally, we propose a
two-stage mapping rule optimization strategy that initially optimizes mapping
rules before querying target LLMs to enhance the efficiency of AutoBreach.
AutoBreach can efficiently identify security vulnerabilities across various
LLMs, including three proprietary models: Claude-3, GPT-3.5, GPT-4 Turbo, and
two LLMs' web platforms: Bingchat, GPT-4 Web, achieving an average success rate
of over 80% with fewer than 10 queries
