  Large Language Models (LLMs) represent an advanced evolution of earlier,
simpler language models. They boast enhanced abilities to handle complex
language patterns and generate coherent text, images, audios, and videos.
Furthermore, they can be fine-tuned for specific tasks. This versatility has
led to the proliferation and extensive use of numerous commercialized large
models. However, the rapid expansion of LLMs has raised security and ethical
concerns within the academic community. This emphasizes the need for ongoing
research into security evaluation during their development and deployment. Over
the past few years, a substantial body of research has been dedicated to the
security evaluation of large-scale models. This article an in-depth review of
the most recent advancements in this field, providing a comprehensive analysis
of commonly used evaluation metrics, advanced evaluation frameworks, and the
routine evaluation processes for LLMs. Furthermore, we also discuss the future
directions for advancing the security evaluation of LLMs.
