  As the development of large language models (LLMs) rapidly advances, securing
these models effectively without compromising their utility has become a
pivotal area of research. However, current defense strategies against jailbreak
attacks (i.e., efforts to bypass security protocols) often suffer from limited
adaptability, restricted general capability, and high cost. To address these
challenges, we introduce SafeAligner, a methodology implemented at the decoding
stage to fortify defenses against jailbreak attacks. We begin by developing two
specialized models: the Sentinel Model, which is trained to foster safety, and
the Intruder Model, designed to generate riskier responses. SafeAligner
leverages the disparity in security levels between the responses from these
models to differentiate between harmful and beneficial tokens, effectively
guiding the safety alignment by altering the output token distribution of the
target model. Extensive experiments show that SafeAligner can increase the
likelihood of beneficial tokens, while reducing the occurrence of harmful ones,
thereby ensuring secure alignment with minimal loss to generality.
