Understanding and improving written peer review for grant applicants.

Written by Jonathan Best of the Wellcome Trust, from the funders perspective.

Peer review, the use of experts to assess the merits of research proposals, forms the backbone of decision-making for research funding. The main role of these reviews is to inform funders decisions regarding which projects to fund by giving expert opinion on the strengths and weaknesses of the proposals compared to the funders criteria.

Published studies have shown that at the very least, reviewers (whether as committees or through written contributions) collectively are able to agree on ‘strong’ proposals but the nuances of good versus better are far more difficult to reach a consensus on (Graves 2011). This aspect of decision-making often means that the order in which projects appear just above or below the funding line varies between parallel committees.

Other analyses comparing publication related metrics, career or social network indicators of applicants from either side of the funding line have shown that in some cases those who initially just miss the funding cut can often go on to do as well as if not better in these measures than their counterparts who were successful at gaining the initial funding (Klaus 2019; Wang 2019). This has led to the suggestion that resilience of the applicant plays a large part in ultimately attaining funding and success can be found through persevering with an idea which at first was rejected (the Derrick hypothesis).

This project

There have been numerous reviews and editorials looking at peer review usually with a focus on the problems with the process. Very few studies have looked at the quality and utility of written feedback to applicants.

We have teamed up with Gemma Derrick’s group at Lancaster University to understand if, how and where providing written peer review feedback to applicants sends constructive signals to researchers that although not funded their proposal is worth pursuing further resulting in future success.  

Bibliography

Graves N, Barnett A, Clarke P. (2011) Funding grant proposals for scientific research: retrospective analysis of scores by members of grant review panel. BMJ. 343: d4797. Published online 2011 Sep 27. doi: 10.1136/bmj.d4797

Klaus B, del Alamo D. (2019) Talent Identification at the limits of Peer Review: an analysis of the EMBO Postdoctoral Fellowships Selection Process. Pre-print doi: https://doi.org/10.1101/481655

Wang Y, Jones BF, Wang D. (2019) Early-career setback and future career impact. Nat Commun. 10(1):4331. doi: 10.1038/s41467-019-12189-3.

Published by Helen Greaves

Helen is currently completing her PhD research which aims to assess the value of pond management for biodiversity conservation. Her work focuses on macroinvertebrate community assemblages and water chemistry analysis. Helen acts as secretary and as a co-organiser of the Norfolk Ponds Project meetings and sits as a student member on the European Pond Conservation Network’s steering committee.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: