  Automatic program repair (APR) aims to reduce the cost of manually fixing
software defects. However, APR suffers from generating a multitude of
overfitting patches, those patches that fail to correctly repair the defect
beyond making the tests pass. This paper presents a novel overfitting patch
detection system called ODS to assess the correctness of APR patches. ODS first
statically compares a patched program and a buggy program in order to extract
code features at the abstract syntax tree (AST) level. Then, ODS uses
supervised learning with the captured code features and patch correctness
labels to automatically learn a probabilistic model. The learned ODS model can
then finally be applied to classify new and unseen program repair patches. We
conduct a large-scale experiment to evaluate the effectiveness of ODS on patch
correctness classification based on 10,302 patches from Defects4J, Bugs.jar and
Bears benchmarks. The empirical evaluation shows that ODS is able to correctly
classify 71.9% of program repair patches from 26 projects, which improves the
state-of-the-art. ODS is applicable in practice and can be employed as a
post-processing procedure to classify the patches generated by different APR
systems.
