Risk evaluation in peer review of grant applications

Stephen Gallo; Lisa Thompson; Karen Schmaling; Scott Glisson

doi:10.1007/s10669-018-9677-6

Back

Risk evaluation in peer review of grant applications

Journal article

Open access

Peer reviewed

Risk evaluation in peer review of grant applications

Stephen Gallo, Lisa Thompson, Karen Schmaling and Scott Glisson

Environment systems & decisions, Vol.38(2), pp.216-229

06/2018

DOI: https://doi.org/10.1007/s10669-018-9677-6

Handle:

https://hdl.handle.net/2376/108800

Files and links (1)

url

https://doi.org/10.1007/s10669-018-9677-6View

Published (Version of record) Open

Abstract

Environment, general

Innovation

Survey

Grant applications

Systems Theory, Control

Research funding

Science, multidisciplinary

Quality Control, Reliability, Safety and Risk

Environment

Peer review

Environmental Management

Novelty

The process of peer review is used to identify the most scientifically meritorious research projects for funding. Impact and innovation are among the criteria used to determine overall merit. A criticism of peer review has been the perception that reviewers are biased against innovation, such as one study that found reviewers to systematically assign poorer scores to highly novel work. Moreover, reviewers’ definitions for excellent research and paradigm-shifting research are different; innovative research may not always be considered excellent. Therefore, it is clear more needs to be done to understand the decision-making processes of reviewers in evaluating risk and innovation in research. In an effort to address this gap, the American Institute of Biological Sciences developed a comprehensive peer review survey that examined, in part, the differences in applicant and reviewer perceptions of review outcomes. The survey was disseminated to 13,091 reviewers and applicants, of whom 9.4% responded. Only 24% of respondent applicants indicated that innovation was addressed in their review feedback, while 81% of respondent reviewers indicated they factored innovation into selecting the best science and 73% viewed innovation as an essential component of scientific excellence. Similarly, while only 27% of respondent applicants reported receiving comments on the riskiness of their grant applications, 58% of respondent reviewers indicated that the risks associated with innovative research impacted the scores they assigned to the grant applications. These results indicate a potential source of bias in how innovation and risk are evaluated in grant applications.

Metrics

9 Record Views

See more details

Details

Title: Risk evaluation in peer review of grant applications
Creators: Stephen Gallo - 0000 0000 9268 7452 grid.299916.c American Institute of Biological Sciences, Scientific Peer Advisory and Review Services 1313 Dolley Madison Blvd. Suite 402 Mclean VA 22101 USA
Lisa Thompson - 0000 0000 9268 7452 grid.299916.c American Institute of Biological Sciences, Scientific Peer Advisory and Review Services 1313 Dolley Madison Blvd. Suite 402 Mclean VA 22101 USA
Karen Schmaling - 0000 0001 2157 6568 grid.30064.31 Washington State University 14204 NE Salmon Creek Avenue Vancouver WA 98686 USA
Scott Glisson - 0000 0000 9268 7452 grid.299916.c American Institute of Biological Sciences, Scientific Peer Advisory and Review Services 1313 Dolley Madison Blvd. Suite 402 Mclean VA 22101 USA
Contributors: Benjamin D Trump (Editor)
Danail Hristozov (Editor)
Igor Linkov (Editor)
Publication Details: Environment systems & decisions, Vol.38(2), pp.216-229
Academic Unit: Psychology, Department of
Publisher: Springer US; New York
Identifiers: 99900546933701842
Language: English
Resource Type: Journal article