  Benchmarks of bugs are essential to empirically evaluate automatic program
repair tools. In this paper, we present Bears, a project for collecting and
storing bugs into an extensible bug benchmark for automatic repair studies in
Java. The collection of bugs relies on commit building state from Continuous
Integration (CI) to find potential pairs of buggy and patched program versions
from open-source projects hosted on GitHub. Each pair of program versions
passes through a pipeline where an attempt of reproducing a bug and its patch
is performed. The core step of the reproduction pipeline is the execution of
the test suite of the program on both program versions. If a test failure is
found in the buggy program version candidate and no test failure is found in
its patched program version candidate, a bug and its patch were successfully
reproduced. The uniqueness of Bears is the usage of CI (builds) to identify
buggy and patched program version candidates, which has been widely adopted in
the last years in open-source projects. This approach allows us to collect bugs
from a diversity of projects beyond mature projects that use bug tracking
systems. Moreover, Bears was designed to be publicly available and to be easily
extensible by the research community through automatic creation of branches
with bugs in a given GitHub repository, which can be used for pull requests in
the Bears repository. We present in this paper the approach employed by Bears,
and we deliver the version 1.0 of Bears, which contains 251 reproducible bugs
collected from 72 projects that use the Travis CI and Maven build environment.
