  Sampling diverse programs from a code language model and reranking with model
likelihood is a popular method for code generation but it is prone to
preferring degenerate solutions. Inspired by collaborative programming, we
propose Coder-Reviewer reranking. We augment Coder language models from past
work, which generate programs given language instructions, with Reviewer
models, which evaluate the likelihood of the instruction given the generated
programs. We perform an extensive study across six datasets with eight models
from three model families. Experimental results show that Coder-Reviewer
reranking leads to consistent and significant improvement (up to 17% absolute
accuracy gain) over reranking with the Coder model only. When combined with
executability filtering, Coder-Reviewer reranking can often outperform the
minimum Bayes risk method. Coder-Reviewer reranking is easy to implement by
prompting, can generalize to different programming languages, and works well
with off-the-shelf hyperparameters.
