  Large Language Models (LLMs) have drawn widespread attention and research due
to their astounding performance in tasks such as text generation and reasoning.
Derivative products, like ChatGPT, have been extensively deployed and highly
sought after. Meanwhile, the evaluation and optimization of LLMs in software
engineering tasks, such as code generation, have become a research focus.
However, there is still a lack of systematic research on the application and
evaluation of LLMs in the field of software engineering. Therefore, this paper
is the first to comprehensively investigate and collate the research and
products combining LLMs with software engineering, aiming to answer two
questions: (1) What are the current integrations of LLMs with software
engineering? (2) Can LLMs effectively handle software engineering tasks? To
find the answers, we have collected related literature as extensively as
possible from seven mainstream databases, and selected 123 papers for analysis.
We have categorized these papers in detail and reviewed the current research
status of LLMs from the perspective of seven major software engineering tasks,
hoping this will help researchers better grasp the research trends and address
the issues when applying LLMs. Meanwhile, we have also organized and presented
papers with evaluation content to reveal the performance and effectiveness of
LLMs in various software engineering tasks, providing guidance for researchers
and developers to optimize.
