  The rapid evolution of Large Language Models (LLMs) has rendered them
indispensable in modern society. While security measures are typically to align
LLMs with human values prior to release, recent studies have unveiled a
concerning phenomenon named "Jailbreak". This term refers to the unexpected and
potentially harmful responses generated by LLMs when prompted with malicious
questions. Most existing research focus on generating jailbreak prompts but
system message configurations vary significantly in experiments. In this paper,
we aim to answer a question: Is the system message really important for
jailbreaks in LLMs? We conduct experiments in mainstream LLMs to generate
jailbreak prompts with varying system messages: short, long, and none. We
discover that different system messages have distinct resistances to
jailbreaks. Therefore, we explore the transferability of jailbreaks across LLMs
with different system messages. Furthermore, we propose the System Messages
Evolutionary Algorithm (SMEA) to generate system messages that are more
resistant to jailbreak prompts, even with minor changes. Through SMEA, we get a
robust system messages population with little change in the length of system
messages. Our research not only bolsters LLMs security but also raises the bar
for jailbreaks, fostering advancements in this field of study.
