It’s hard to be in academia without hearing about the dreaded Reviewer #3. Despite the thousands of jokes, very little is actually known about the impact of the review process on researchers – and on the research community!
Competitive funding is a core component of maintaining a thriving and innovative research culture – but the focus has long been on the individual instead of the collective. With this individual lens, it is not uncommon for researchers to see funding as a zero-sum game, after all, there is a limited amount of funding available.
Whilst this may be a reasonable response for an individual, it should not be the perspective of the funding systems. Any country benefits immensely from an empowered scientific workforce that produces increasingly competitive research proposals – so why is peer review so focused on tearing down instead of listing up?
At Proposal Analytics, we propose a research review process that generates constructive comments that can help the applicant grow, regardless of the funding decision. Scientific gatekeeping[1], the use of impersonal and biased metrics[2][3], and a needlessly unpleasant reviewer #3 do not make the scientific enterprise better – instead they only reduce the potential achievements we can reach. A more collaborative view on reviewing; one that is still highly critical yet constructive and created with the intention of empowering researchers to improve with each iteration, is one in which we can reach our collective potential.
We are collaborating on this project with the Wellcome Trust and Gemma Derrick at Lancaster University because we are interested in studying how reviewer comments impact the research ecosystem, and specifically how critical comments can be used to improve the collective intelligence of early career researchers.
Proposal Analytics already looks into the grant submission patterns of researchers in the US – where do they submit and how that breaks down across racial and gender groups. With this project, we are interested in understanding if the reviewer comments help or hinder researchers’ future grant applications. Do researchers resubmit to the same funding source after improving their application, or are they disheartened by harsh criticisms and seek another – perhaps more compassionate – funder? If so, can review comments therefore be constructed in a way that doesn’t dishearten the researcher, but instead motivates their improvement?
Answering these questions will help us evaluate how effective the current funding infrastructure is at cultivating a competitive and innovative research ecosystem that includes and improves its participants, not alienates them.
[1] https://advances.sciencemag.org/content/6/23/eaaz4868
[3] https://www.nature.com/news/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351
Photo by “My Life Through A Lens” on Unsplash
Subscribe to other blog posts on this subject by adding your email address below:
pearlaboratory.org 