  Benefiting from the powerful capabilities of Large Language Models (LLMs),
pre-trained visual encoder models connected to an LLMs can realize Vision
Language Models (VLMs). However, existing research shows that the visual
modality of VLMs is vulnerable, with attackers easily bypassing LLMs' safety
alignment through visual modality features to launch attacks. To address this
issue, we enhance the existing VLMs' visual modality safety alignment by adding
safety modules, including a safety projector, safety tokens, and a safety head,
through a two-stage training process, effectively improving the model's defense
against risky images. For example, building upon the LLaVA-v1.5 model, we
achieve a safety score of 8.26, surpassing the GPT-4V on the Red Teaming Visual
Language Models (RTVLM) benchmark. Our method boasts ease of use, high
flexibility, and strong controllability, and it enhances safety while having
minimal impact on the model's general performance. Moreover, our alignment
strategy also uncovers some possible risky content within commonly used
open-source multimodal datasets. Our code will be open sourced after the
anonymous review.
