  Previous studies have shown that Automated Program Repair (APR) techniques
suffer from the overfitting problem. Overfitting happens when a patch is run
and the test suite does not reveal any error, but the patch actually does not
fix the underlying bug or it introduces a new defect that is not covered by the
test suite. Therefore, the patches generated by APR tools need to be validated
by human programmers, which can be very costly, and prevents APR tools adoption
in practice.Our work aims at increasing developer trust in automated patch
generation by minimizing the number of plausible patches that they have to
review, thereby reducing the time required to find a correct patch. We
introduce a novel light-weight test-based patch clustering approach called
xTestCluster, which clusters patches based on their dynamic behavior.
xTestCluster is applied after the patch generation phase in order to analyze
the generated patches from one or more repair tools. The novelty of
xTestCluster lies in using information from execution of newly generated test
cases to cluster patches generated by multiple APR approaches. A cluster is
formed with patches that fail on the same generated test cases. The output from
xTestCluster gives developers a) a way of reducing the number of patches to
analyze, as they can focus on analyzing a sample of patches from each cluster,
b) additional information attached to each patch. After analyzing 1910
plausible patches from 25 Java APR tools, our results show that xTestCluster is
able to reduce the number of patches to review and analyze with a median of
50%. xTestCluster can save a significant amount of time for developers that
have to review the multitude of patches generated by APR tools, and provides
them with new test cases that show the differences in behavior between
generated patches.
