  Third-party dependency updates can cause a build to fail if the new
dependency version introduces a change that is incompatible with the usage:
this is called a breaking dependency update. Research on breaking dependency
updates is active, with works on characterization, understanding, automatic
repair of breaking updates, and other software engineering aspects. All such
research projects require a benchmark of breaking updates that has the
following properties: 1) it contains real-world breaking updates; 2) the
breaking updates can be executed; 3) the benchmark provides stable scientific
artifacts of breaking updates over time, a property we call reproducibility. To
the best of our knowledge, such a benchmark is missing. To address this
problem, we present BUMP, a new benchmark that contains reproducible breaking
dependency updates in the context of Java projects built with the Maven build
system. BUMP contains 571 breaking dependency updates collected from 153 Java
projects. BUMP ensures long-term reproducibility of dependency updates on
different platforms, guaranteeing consistent build failures. We categorize the
different causes of build breakage in BUMP, providing novel insights for future
work on breaking update engineering. To our knowledge, BUMP is the first of its
kind, providing hundreds of real-world breaking updates that have all been made
reproducible.
