  Large Language Models (LLMs) are widely used in natural language processing
but face the risk of jailbreak attacks that maliciously induce them to generate
harmful content. Existing jailbreak attacks, including character-level and
context-level attacks, mainly focus on the prompt of the plain text without
specifically exploring the significant influence of its structure. In this
paper, we focus on studying how prompt structure contributes to the jailbreak
attack. We introduce a novel structure-level attack method based on tail
structures that are rarely used during LLM training, which we refer to as
Uncommon Text-Encoded Structure (UTES). We extensively study 12 UTESs templates
and 6 obfuscation methods to build an effective automated jailbreak tool named
StructuralSleight that contains three escalating attack strategies: Structural
Attack, Structural and Character/Context Obfuscation Attack, and Fully
Obfuscated Structural Attack. Extensive experiments on existing LLMs show that
StructuralSleight significantly outperforms baseline methods. In particular,
the attack success rate reaches 94.62\% on GPT-4o, which has not been addressed
by state-of-the-art techniques.
