  Ensuring the safety of artificial intelligence-generated content (AIGC) is a
longstanding topic in the artificial intelligence (AI) community, and the
safety concerns associated with Large Language Models (LLMs) have been widely
investigated. Recently, large vision-language models (VLMs) represent an
unprecedented revolution, as they are built upon LLMs but can incorporate
additional modalities (e.g., images). However, the safety of VLMs lacks
systematic evaluation, and there may be an overconfidence in the safety
guarantees provided by their underlying LLMs. In this paper, to demonstrate
that introducing additional modality modules leads to unforeseen AI safety
issues, we propose FigStep, a straightforward yet effective jailbreaking
algorithm against VLMs. Instead of feeding textual harmful instructions
directly, FigStep converts the harmful content into images through typography
to bypass the safety alignment within the textual module of the VLMs, inducing
VLMs to output unsafe responses that violate common AI safety policies. In our
evaluation, we manually review 46,500 model responses generated by 3 families
of the promising open-source VLMs, i.e., LLaVA, MiniGPT4, and CogVLM (a total
of 6 VLMs). The experimental results show that FigStep can achieve an average
attack success rate of 82.50% on 500 harmful queries in 10 topics. Moreover, we
demonstrate that the methodology of FigStep can even jailbreak GPT-4V, which
already leverages an OCR detector to filter harmful queries. Above all, our
work reveals that VLMs are vulnerable to jailbreaking attacks, which highlights
the necessity of novel safety alignments between visual and textual modalities.
