  Unit test is crucial for detecting bugs in individual program units but
consumes time and effort. The existing automated unit test generation methods
are mainly based on search-based software testing (SBST) and language models to
liberate developers. Recently, large language models (LLMs) have demonstrated
remarkable reasoning and generation capabilities. However, several problems
limit their ability to generate high-quality test cases: (1) LLMs may generate
invalid test cases under insufficient context, resulting in compilation errors;
(2) Lack of test and coverage feedback information may cause runtime errors and
low coverage rates. (3) The repetitive suppression problem causes LLMs to get
stuck into the repetition loop of self-repair or re-generation attempts. In
this paper, we propose TestART, a novel unit test generation method that
leverages the strengths of LLMs while overcoming the limitations mentioned.
TestART improves LLM-based unit test via co-evolution of automated generation
and repair iteration. TestART leverages the template-based repair technique to
fix bugs in LLM-generated test cases, using prompt injection to guide the
next-step automated generation and avoid repetition suppression. Furthermore,
TestART extracts coverage information from the passed test cases and utilizes
it as testing feedback to enhance the sufficiency of the final test case. This
synergy between generation and repair elevates the quality, effectiveness, and
readability of the produced test cases significantly beyond previous methods.
In comparative experiments, the pass rate of TestART-generated test cases is
78.55%, which is approximately 18% higher than both the ChatGPT-4.0 model and
the same ChatGPT-3.5-based method ChatUniTest. It also achieves an impressive
line coverage rate of 90.96% on the focal methods that passed the test,
exceeding EvoSuite by 3.4%.
