  Peer review is an integral component of contemporary science. While peer
review focuses attention on promising and interesting science, it also
encourages scientists to pursue some questions at the expense of others. Here,
we use ideas from forecasting assessment to examine how two modes of peer
review -- ex ante review of proposals for future work and ex post review of
completed science -- motivate scientists to favor some questions instead of
others. Our main result is that ex ante and ex post peer review push
investigators toward distinct sets of scientific questions. This tension arises
because ex post review allows an investigator to leverage her own scientific
beliefs to generate results that others will find surprising, whereas ex ante
review does not. Moreover, ex ante review will favor different research
questions depending on whether reviewers rank proposals in anticipation of
changes to their own personal beliefs, or to the beliefs of their peers. The
tension between ex ante and ex post review puts investigators in a bind,
because most researchers need to find projects that will survive both. By
unpacking the tension between these two modes of review, we can understand how
they shape the landscape of science and how changes to peer review might shift
scientific activity in unforeseen directions.
