  Large Language Models have gained considerable attention for their
revolutionary capabilities. However, there is also growing concern on their
safety implications, making a comprehensive safety evaluation for LLMs urgently
needed before model deployment. In this work, we propose S-Eval, a new
comprehensive, multi-dimensional and open-ended safety evaluation benchmark. At
the core of S-Eval is a novel LLM-based automatic test prompt generation and
selection framework, which trains an expert testing LLM Mt combined with a
range of test selection strategies to automatically construct a high-quality
test suite for the safety evaluation. The key to the automation of this process
is a novel expert safety-critique LLM Mc able to quantify the riskiness score
of an LLM's response, and additionally produce risk tags and explanations.
Besides, the generation process is also guided by a carefully designed risk
taxonomy with four different levels, covering comprehensive and
multi-dimensional safety risks of concern. Based on these, we systematically
construct a new and large-scale safety evaluation benchmark for LLMs consisting
of 220,000 evaluation prompts, including 20,000 base risk prompts (10,000 in
Chinese and 10,000 in English) and 200,000 corresponding attack prompts derived
from 10 popular adversarial instruction attacks against LLMs. Moreover,
considering the rapid evolution of LLMs and accompanied safety threats, S-Eval
can be flexibly configured and adapted to include new risks, attacks and
models. S-Eval is extensively evaluated on 20 popular and representative LLMs.
The results confirm that S-Eval can better reflect and inform the safety risks
of LLMs compared to existing benchmarks. We also explore the impacts of
parameter scales, language environments, and decoding parameters on the
evaluation, providing a systematic methodology for evaluating the safety of
LLMs.
