  Many publicly available language models have been safety tuned to reduce the
likelihood of toxic or liability-inducing text. Users or security analysts
attempt to jailbreak or redteam these models with adversarial prompts which
cause compliance with requests. One attack method is to apply discrete
optimization techniques to the prompt. However, the resulting attack strings
are often gibberish text, easily filtered by defenders due to high measured
perplexity, and may fail for unseen tasks and/or well-tuned models. In this
work, we improve existing algorithms (primarily GCG and BEAST) to develop
powerful and fluent attacks on safety-tuned models like Llama-2 and Phi-3. Our
technique centers around a new distillation-based approach that encourages the
victim model to emulate a toxified finetune, either in terms of output
probabilities or internal activations. To encourage human-fluent attacks, we
add a multi-model perplexity penalty and a repetition penalty to the objective.
We also enhance optimizer strength by allowing token insertions, token swaps,
and token deletions and by using longer attack sequences. The resulting process
is able to reliably jailbreak the most difficult target models with prompts
that appear similar to human-written prompts. On Advbench we achieve attack
success rates $>93$% for Llama-2-7B, Llama-3-8B, and Vicuna-7B, while
maintaining model-measured perplexity $<33$; we achieve $95$% attack success
for Phi-3, though with higher perplexity. We also find a universally-optimized
single fluent prompt that induces $>88$% compliance on previously unseen tasks
across Llama-2-7B, Phi-3-mini and Vicuna-7B and transfers to other black-box
models.
