  Context: The SZZ algorithm is the de facto standard for labeling bug fixing
commits and finding inducing changes for defect prediction data. Recent
research uncovered potential problems in different parts of the SZZ algorithm.
Most defect prediction data sets provide only static code metrics as features,
while research indicates that other features are also important.
  Objective: We provide an empirical analysis of the defect labels created with
the SZZ algorithm and the impact of commonly used features on results.
  Method: We used a combination of manual validation and adopted or improved
heuristics for the collection of defect data. We conducted an empirical study
on 398 releases of 38 Apache projects.
  Results: We found that only half of the bug fixing commits determined by SZZ
are actually bug fixing. If a six-month time frame is used in combination with
SZZ to determine which bugs affect a release, one file is incorrectly labeled
as defective for every file that is correctly labeled as defective. In
addition, two defective files are missed. We also explored the impact of the
relatively small set of features that are available in most defect prediction
data sets, as there are multiple publications that indicate that, e.g., churn
related features are important for defect prediction. We found that the
difference of using more features is not significant.
  Conclusion: Problems with inaccurate defect labels are a severe threat to the
validity of the state of the art of defect prediction. Small feature sets seem
to be a less severe threat.
