  Misuse of the Large Language Models (LLMs) has raised widespread concern. To
address this issue, safeguards have been taken to ensure that LLMs align with
social ethics. However, recent findings have revealed an unsettling
vulnerability bypassing the safeguards of LLMs, known as jailbreak attacks. By
applying techniques, such as employing role-playing scenarios, adversarial
examples, or subtle subversion of safety objectives as a prompt, LLMs can
produce an inappropriate or even harmful response. While researchers have
studied several categories of jailbreak attacks, they have done so in
isolation. To fill this gap, we present the first large-scale measurement of
various jailbreak attack methods. We concentrate on 13 cutting-edge jailbreak
methods from four categories, 160 questions from 16 violation categories, and
six popular LLMs. Our extensive experimental results demonstrate that the
optimized jailbreak prompts consistently achieve the highest attack success
rates, as well as exhibit robustness across different LLMs. Some jailbreak
prompt datasets, available from the Internet, can also achieve high attack
success rates on many LLMs, such as ChatGLM3, GPT-3.5, and PaLM2. Despite the
claims from many organizations regarding the coverage of violation categories
in their policies, the attack success rates from these categories remain high,
indicating the challenges of effectively aligning LLM policies and the ability
to counter jailbreak attacks. We also discuss the trade-off between the attack
performance and efficiency, as well as show that the transferability of the
jailbreak prompts is still viable, becoming an option for black-box models.
Overall, our research highlights the necessity of evaluating different
jailbreak methods. We hope our study can provide insights for future research
on jailbreak attacks and serve as a benchmark tool for evaluating them for
practitioners.
