  Autonomous program improvement typically involves automatically producing bug
fixes and feature additions. Such program improvement can be accomplished by a
combination of large language model (LLM) and program analysis capabilities, in
the form of an LLM agent. Since program repair or program improvement typically
requires a specification of intended behavior - specification inference can be
useful for producing high quality program patches. In this work, we examine
efficient and low-cost workflows for iterative specification inference within
an LLM agent. Given a GitHub issue to be resolved in a software project, our
goal is to conduct iterative code search accompanied by specification inference
- thereby inferring intent from both the project structure and behavior. The
intent thus captured is examined by a reviewer agent with the goal of vetting
the patches as well as providing a measure of confidence in the vetted patches.
Our approach SpecRover (AutoCodeRover-v2) is built on the open-source LLM agent
AutoCodeRover. In an evaluation on the full SWE-Bench consisting of 2294 GitHub
issues, it shows more than 50% improvement in efficacy over AutoCodeRover.
Compared to the open-source agents available, our work shows modest cost ($0.65
per issue) in resolving an average GitHub issue in SWE-Bench lite. The
production of explanation by SpecRover allows for a better "signal" to be given
to the developer, on when the suggested patches can be accepted with
confidence. SpecRover also seeks to demonstrate the continued importance of
specification inference in automated program repair, even as program repair
technologies enter the LLM era.
