Abstract
Purpose
Since peer review for funding decisions is crucial to the scientific system, we direct the reader towards new ideas related to research funding and the associated peer review process.
Design/methodology/approach
We describe the overall structure of the funding review system and explore the expectations of its various key stakeholders. An examination of testing across the review processes of different funding agencies revealed several issues in the current system. We then summarize the efforts to explore potential solutions. Before concluding, we also discuss recent initiatives, including partial lottery mechanisms, distributed peer review, and methods for identifying originality in proposals by examining areas of non-consensus among reviewers and applicants.
Findings
It is difficult to test whether the funding peer review system functions as expected. Moreover, when the peer-review process was replicated across different review groups, the inter-rater problem, where two or more well-intentioned reviewers reached divergent conclusions, was found to be widespread in funding evaluations. At its core, this issue stems from substantive disagreements among reviewers, which can introduce bias into the process. As a result, organizing a peer-review system that is fair, valid, and reliable for funding decisions is particularly challenging. The contemporary organization of the funding review system does not guarantee that it can fulfill its purpose. Consequently, scientists are looking to substantiate funding decisions with more scientific evidence. Some new initiatives have been proposed, which are either more interactive with a strictly organized procedure or are more random (or stochastic), leading to less bias.
Research limitations
For practical reasons, we were not able to discuss all, or at least the main, funders in the world.
Practical implications
Considering the various steps in peer review procedures for funding decisions may inspire the readers to suggest improvements to the existing system, resulting in reduced bias and greater equality among scientists.
Originality/value
Our work contributes to understanding peer review in funding contexts and to exploring possible reforms aimed at improving the existing system.