  Pull requests are a key part of the collaborative software development and
code review process today. However, pull requests can also slow down the
software development process when the reviewer(s) or the author do not actively
engage with the pull request. In this work, we design an end-to-end service,
Nudge, for accelerating overdue pull requests towards completion by reminding
the author or the reviewer(s) to engage with their overdue pull requests.
First, we use models based on effort estimation and machine learning to predict
the completion time for a given pull request. Second, we use activity detection
to filter out pull requests that may be overdue, but for which sufficient
action is taking place nonetheless. Lastly, we use actor identification to
understand who the blocker of the pull request is and nudge the appropriate
actor (author or reviewer(s)). The key novelty of Nudge is that it succeeds in
reducing pull request resolution time, while ensuring that developers perceive
the notifications sent as useful, at the scale of thousands of repositories. In
a randomized trial on 147 repositories in use at Microsoft, Nudge was able to
reduce pull request resolution time by 60% for 8,500 pull requests, when
compared to overdue pull requests for which Nudge did not send a notification.
Furthermore, developers receiving Nudge notifications resolved 73% of these
notifications as positive. We observed similar results when scaling up the
deployment of Nudge to 8,000 repositories at Microsoft, for which Nudge sent
210,000 notifications during a full year. This demonstrates Nudge's ability to
scale to thousands of repositories. Lastly, our qualitative analysis of a
selection of Nudge notifications indicates areas for future research, such as
taking dependencies among pull requests and developer availability into
account.
