  Jailbreaking is an emerging adversarial attack that bypasses the safety
alignment deployed in off-the-shelf large language models (LLMs) and has
evolved into four major categories: optimization-based attacks such as Greedy
Coordinate Gradient (GCG), jailbreak template-based attacks such as
"Do-Anything-Now", advanced indirect attacks like DrAttack, and multilingual
jailbreaks. However, delivering a practical jailbreak defense is challenging
because it needs to not only handle all the above jailbreak attacks but also
incur negligible delay to user prompts, as well as be compatible with both
open-source and closed-source LLMs.
  Inspired by how the traditional security concept of shadow stacks defends
against memory overflow attacks, this paper introduces a generic LLM jailbreak
defense framework called SelfDefend, which establishes a shadow LLM defense
instance to concurrently protect the target LLM instance in the normal stack
and collaborate with it for checkpoint-based access control. The effectiveness
of SelfDefend builds upon our observation that existing LLMs (both target and
defense LLMs) have the capability to identify harmful prompts or intentions in
user queries, which we empirically validate using the commonly used GPT-3.5/4
models across all major jailbreak attacks. Our measurements show that
SelfDefend enables GPT-3.5 to suppress the attack success rate (ASR) by
8.97-95.74% (average: 60%) and GPT-4 by even 36.36-100% (average: 83%), while
incurring negligible effects on normal queries. To further improve the
defense's robustness and minimize costs, we employ a data distillation approach
to tune dedicated open-source defense models. These models outperform four SOTA
defenses and match the performance of GPT-4-based SelfDefend, with
significantly lower extra delays. We also empirically show that the tuned
models are robust to targeted GCG and prompt injection attacks.
