National Bureau of Economic Research, Inc.

To investigate and experiment with alternative peer-review methods for selecting among scientific research proposals

  • Amount $725,614
  • City Cambridge, MA
  • Investigator Paula Stephan
  • Initiative Economic Analysis of Science and Technology (EAST)
  • Year 2021
  • Program Research
  • Sub-program Economic Institutions, Behavior, & Performance

Funding decisions by the National Science Foundation and the National Institutes of Health are based on a process of peer review where independent subject-matter experts are invited to rate and rank potential research projects along a number of criteria, including the importance of the issue being examined, the strength of the experimental design, the likelihood of success, and the qualifications of the research team.  Projects that review well are funded.  Those that review poorly are not.  The budgets of just those two institutions total some $50 billion dollars per year—the lion’s share of basic research funding in the U.S.—so a lot rides on whether the peer review process is a good way to identify and select promising research projects.  This grant funds a series of projects led by Paula Stephan, Professor of Economics at Georgia State University and Research Associate at the National Bureau of Economic Research and Chiara Franzoni, Associate Professor at the School of Management at Milan Polytechnic University, to rigorously study peer review when used as a grant allocation process.  Stephan and Franzoni will partner with Novo Nordisk, a Danish foundation that has been funding research to combat diabetes since 1923 and that has kept meticulous records of its peer review and funding decisions.  This unique longitudinal dataset will allow the research team to compare the initial judgements of reviewers with the eventual successes of both funded and unfunded projects.  Second, Stephan and Franzoni have designed a series of experiments that will set up peer review panels with different decision structures and racial and gender compositions and give each panel the same set of research proposals to rate, giving the team some evidence of how structure and composition affect the verdicts of peer review committees.  Third, the team has assembled a small cohort of foundations that are willing to experiment with doing away with parts of the peer review altogether, introducing randomness at different stages of the selection process from a pool of proposals that meet certain base quality criteria.  These three separate initiatives will be supplemented alongside a host of qualitative and quantitative interviews that probe experts about the peer review process and their own judgements about its efficacy.

Back to grants database
We use cookies to analyze our traffic. Please decide if you are willing to accept cookies from our website.