  Context: Due to the association of significant efforts, even a minor
improvement in the effectiveness of Code Reviews(CR) can incur significant
savings for a software development organization. Aim: This study aims to
develop a finer grain understanding of what makes a code review comment useful
to OSS developers, to what extent a code review comment is considered useful to
them, and how various contextual and participant-related factors influence its
usefulness level. Method: On this goal, we have conducted a three-stage
mixed-method study. We randomly selected 2,500 CR comments from the OpenDev
Nova project and manually categorized the comments. We designed a survey of
OpenDev developers to better understand their perspectives on useful CRs.
Combining our survey-obtained scores with our manually labeled dataset, we
trained two regression models - one to identify factors that influence the
usefulness of CR comments and the other to identify factors that improve the
odds of `Functional' defect identification over the others. Key findings: The
results of our study suggest that a CR comment's usefulness is dictated not
only by its technical contributions such as defect findings or quality
improvement tips but also by its linguistic characteristics such as
comprehensibility and politeness. While a reviewer's coding experience
positively associates with CR usefulness, the number of mutual reviews, comment
volume in a file, the total number of lines added /modified, and CR interval
has the opposite associations. While authorship and reviewership experiences
for the files under review have been the most popular attributes for reviewer
recommendation systems, we do not find any significant association of those
attributes with CR usefulness.
