  The constant demand for new features and bug fixes are forcing software
projects to shorten cycles and deliver updates ever faster, while sustaining
software quality. The availability of inexpensive, virtualized, cloud-computing
has helped shorten schedules, by enabling continuous integration (CI) on
demand. Platforms like GitHub support CI in-the-cloud. In projects using CI, a
user submitting a pull request triggers a CI step. Besides speeding up build
and test, this fortuitously creates voluminous archives of build and test
successes and failures. CI is a relatively new phenomenon, and these archives
allow a detailed study of CI. How many problems are exposed? Where do they
occur? What factors affect CI failures? Does the "initial quality" as
ascertained by CI predict how many bugs will later appear ("eventual quality")
in the code? In this paper, we undertake a large-scale, fine resolution study
of these records, to better understand CI processes, the nature, and predictors
of CI failures, and the relationship of CI failures to the eventual quality of
the code. We find that: a) CI failures appear to be concentrated in a few
files, just like normal bugs; b) CI failures are not very highly correlated
with eventual failures; c) The use of CI in a pull request doesn't necessarily
mean the code in that request is of good quality.
