  Understanding the root cause of a defect is critical to isolating and
repairing buggy behavior. We present Causal Testing, a new method of root-cause
analysis that relies on the theory of counterfactual causality to identify a
set of executions that likely hold key causal information necessary to
understand and repair buggy behavior. Using the Defects4J benchmark, we find
that Causal Testing could be applied to 71% of real-world defects, and for 77%
of those, it can help developers identify the root cause of the defect. A
controlled experiment with 37 developers shows that Causal Testing improves
participants' ability to identify the cause of the defect from 80% of the time
with standard testing tools to 86% of the time with Causal Testing. The
participants report that Causal Testing provides useful information they cannot
get using tools such as JUnit. Holmes, our prototype, open-source Eclipse
plugin implementation of Causal Testing, is available at
http://holmes.cs.umass.edu/.
