  The most natural method for evaluating program repair systems is to run them
on bug datasets, such as Defects4J. Yet, using this evaluation technique on
arbitrary real-world programs requires heavy configuration. In this paper, we
propose a purely static method to evaluate the potential of the search space of
repair approaches. This new method enables researchers and practitioners to
encode the search spaces of repair approaches and select potentially useful
ones without struggling with tool configuration and execution. We encode the
search spaces by specifying the repair strategies they employ. Next, we use the
specifications to check whether past commits lie in repair search spaces. For a
repair approach, including many human-written past commits in its search space
indicates its potential to generate useful patches. We implement our evaluation
method in LighteR. LighteR gets a Git repository and outputs a list of commits
whose source code changes lie in repair search spaces. We run LighteR on 55,309
commits from the history of 72 Github repositories with and show that LighteR's
precision and recall are 77% and 92%, respectively. Overall, our experiments
show that our novel method is both lightweight and effective to study the
search space of program repair approaches.
