  Automated debugging techniques have the potential to reduce developer effort
in debugging, and have matured enough to be adopted by industry. However, one
critical issue with existing techniques is that, while developers want
rationales for the provided automatic debugging results, existing techniques
are ill-suited to provide them, as their deduction process differs
significantly from that of human developers. Inspired by the way developers
interact with code when debugging, we propose Automated Scientific Debugging
(AutoSD), a technique that given buggy code and a bug-revealing test, prompts
large language models to automatically generate hypotheses, uses debuggers to
actively interact with buggy code, and thus automatically reach conclusions
prior to patch generation. By aligning the reasoning of automated debugging
more closely with that of human developers, we aim to produce intelligible
explanations of how a specific patch has been generated, with the hope that the
explanation will lead to more efficient and accurate developer decisions. Our
empirical analysis on three program repair benchmarks shows that AutoSD
performs competitively with other program repair baselines, and that it can
indicate when it is confident in its results. Furthermore, we perform a human
study with 20 participants, including six professional developers, to evaluate
the utility of explanations from AutoSD. Participants with access to
explanations could judge patch correctness in roughly the same time as those
without, but their accuracy improved for five out of six real-world bugs
studied: 70% of participants answered that they wanted explanations when using
repair tools, while 55% answered that they were satisfied with the Scientific
Debugging presentation.
