Written by Jonathan Best of the Wellcome Trust, from the funders perspective.
Peer review, the use of experts to assess the merits of research proposals, forms the backbone of decision-making for research funding. The main role of these reviews is to inform funders decisions regarding which projects to fund by giving expert opinion on the strengths and weaknesses of the proposals compared to the funders criteria.
Published studies have shown that at the very least, reviewers (whether as committees or through written contributions) collectively are able to agree on ‘strong’ proposals but the nuances of good versus better are far more difficult to reach a consensus on (Graves 2011). This aspect of decision-making often means that the order in which projects appear just above or below the funding line varies between parallel committees.
Other analyses comparing publication related metrics, career or social network indicators of applicants from either side of the funding line have shown that in some cases those who initially just miss the funding cut can often go on to do as well as if not better in these measures than their counterparts who were successful at gaining the initial funding (Klaus 2019; Wang 2019). This has led to the suggestion that resilience of the applicant plays a large part in ultimately attaining funding and success can be found through persevering with an idea which at first was rejected (the Derrick hypothesis).
This project
There have been numerous reviews and editorials looking at peer review usually with a focus on the problems with the process. Very few studies have looked at the quality and utility of written feedback to applicants.
We have teamed up with Gemma Derrick’s group at Lancaster University to understand if, how and where providing written peer review feedback to applicants sends constructive signals to researchers that although not funded their proposal is worth pursuing further resulting in future success.
Bibliography
Graves N, Barnett A, Clarke P. (2011) Funding grant proposals for scientific research: retrospective analysis of scores by members of grant review panel. BMJ. 343: d4797. Published online 2011 Sep 27. doi: 10.1136/bmj.d4797
Klaus B, del Alamo D. (2019) Talent Identification at the limits of Peer Review: an analysis of the EMBO Postdoctoral Fellowships Selection Process. Pre-print doi: https://doi.org/10.1101/481655
Wang Y, Jones BF, Wang D. (2019) Early-career setback and future career impact. Nat Commun. 10(1):4331. doi: 10.1038/s41467-019-12189-3.
pearlaboratory.org 