  Code review is an essential part to software development lifecycle since it
aims at guaranteeing the quality of codes. Modern code review activities
necessitate developers viewing, understanding and even running the programs to
assess logic, functionality, latency, style and other factors. It turns out
that developers have to spend far too much time reviewing the code of their
peers. Accordingly, it is in significant demand to automate the code review
process. In this research, we focus on utilizing pre-training techniques for
the tasks in the code review scenario. We collect a large-scale dataset of
real-world code changes and code reviews from open-source projects in nine of
the most popular programming languages. To better understand code diffs and
reviews, we propose CodeReviewer, a pre-trained model that utilizes four
pre-training tasks tailored specifically for the code review scenario. To
evaluate our model, we focus on three key tasks related to code review
activities, including code change quality estimation, review comment generation
and code refinement. Furthermore, we establish a high-quality benchmark dataset
based on our collected data for these three tasks and conduct comprehensive
experiments on it. The experimental results demonstrate that our model
outperforms the previous state-of-the-art pre-training approaches in all tasks.
Further analysis show that our proposed pre-training tasks and the multilingual
pre-training dataset benefit the model on the understanding of code changes and
reviews.
