  Does past performance influence success in grant applications? In this study
we test whether the grant allocation decisions of the Netherlands Research
Council for the Economic and Social Sciences correlate with the past
performances of the applicants in terms of publications and citations, and with
the results of the peer review process organized by the Council. We show that
the Council is successful in distinguishing grant applicants with above-average
performance from those with below-average performance, but within the former
group no correlation could be found between past performance and receiving a
grant. When comparing the best performing researchers who were denied funding
with the group of researchers who received it, the rejected researchers
significantly outperformed the funded ones. Furthermore, the best rejected
proposals score on average as high on the outcomes of the peer review process
as the accepted proposals. Finally, we found that the Council under study
successfully corrected for gender effects during the selection process. We
explain why these findings may be more general than for this case only.
However, if research councils are not able to select the 'best' researchers,
perhaps they should reconsider their mission. In a final section with policy
implications, we discuss the role of research councils at the level of the
science system in terms of variation, innovation, and quality control.
