Why is feedback a fix for a failed peer review system?

Dr. Gemma Derrick reflects on why the Research Phoenix project is both timely and of paramount importance.

Building a kinder research culture has received a lot of attention of late and there is new hope in changing a system that can be unforgiving, competitive and heartbreaking at the very best of times.  If nothing else, I have previously stated that the COVID-19 pandemic provides a great opportunity to demonstrate kindness, as it is clear in the initial stages of the pandemic that kindness was possible (Derrick, 2020).  For us, normalising failure so it is something that we can (and will) openly acknowledge, and then using it to challenge and change our current peer review system, is part of achieving this kinder, new normal.  

Not so long ago, I had the privilege of mentoring a number of colleagues who were all Early Career Researchers and all applying for the same grant.  They had submitted their applications; meticulously prepared, excellent examples of research proposals and worthy of funding.  However, when the results were released, all were left with the same, stock-standard rejection email. 

Why?  What was the reason, we asked ourselves? 

Nothing.  Silence.  Just the rejection letter, the dreaded sentence that started with ‘unfortunately” and ended with a reassurance that ‘it was an extremely competitive call’ and that ‘more applications were rejected than selected for funding’.  They were in good company it would seem but were lost as to how to progress with otherwise excellent ideas. 

Part of transforming the peer review process requires us (funders, reviewers and researchers alike) to reimagine peer review as something more than providing a binary decision: rejected or accepted.  Instead, we should aim to transform it into a system that we can use to encourage learning, help develop stronger research ideas as well as researchers.  The idea of providing feedback in a way that places applicant needs at the centre of the sense-making of decisions is obvious, but surprisingly it is lacking in current peer review processes.  Many funding agencies do offer a form of feedback dialogue, including giving applicants a chance to response to reviewers’ comments (e.g. UKRI/RCUK).  Others, such as the Wellcome Trust, provide a decision with accompanying rationales which is a step in the right direction. The provision of this feedback is a vital step towards building a kinder research culture, and a better peer review system. 

Feedback is defined as a process where learners make sense of information, and then use it to enhance their work and strategies (Boud & Molloy, 2013; Carless & Boud, 2018)[1].  The provision, or lack thereof, of feedback to enable improvement, is one of the most problematic and frustrating aspects of the academic experience.  Beyond the PhD, the provision of feedback on submitted work – articles, grant and promotion applications – concentrates on providing judgements, outcomes, and not always on providing information that aims to foster the future development of ideas or people.  Under current academic governance structures such as peer review, individuals and their ideas either win, or lose; they are funded, or they are not; they are accepted or rejected.  Here, the outcomes are explained through political or organisational rationales for the decisions, rather than providing individuals with actionable points that they can use.  More importantly, this current system only fulfils one of the aspects of the feedback definition (learners make sense of the information – aka “We are not giving you funding, and this is why”) but fails in the other, arguably more important half of the definition, which is that learners must then ‘use’ the information to enhance their work and future strategies.   

Whereas one might be forgiven for thinking academics already have sufficient personal experience with feedback (and rejection) to be able to implement it effectively, more often than not, the personal reaction we have to the rejection in an ever competitive research world, is one that can blur our ability to make sense of it and act accordingly in the future.  Rejection, and receiving feedback associated with that specific “failure” can still be debilitating, even for seasoned academics.  Our personal experience with rejection and disappointment in the current peer review system does not necessarily result in better science, or more resilient academics, nor is it a catalyst for learning and development.  This is what learning, teaching and assessment experts, refer to as our ‘feedback literacy’.  Within this, our ability to manage our own equilibrium, affects our ability to engage with the feedback as critical commentary (Carless & Boud, 2018), not just as a reason for rejection, and act on it for future funding attempts and/or iterations of the same idea.  For the research community, this can have devastating effects including damage to the mental wellbeing of existing academics, a loss of confidence and the loss of talent if people decide to leave thinking that rejection is a ‘sign’ that they aren’t able to survive in competitive academia.  However, in many situations, scholars, like students, underplay their own agency in actualising their own improvement via the provided feedback (Winstone et al, 2017).  For this reason, the tone of communicating decisions, as well as the lens by which decisions are made, are  vitally important to build a peer review system that doesn’t fail applicants, but instead is kind, by providing useful feedback in a tone that empowers individuals, and doesn’t cast them aside. 

There are still questions surrounding how academics act on feedback, especially when the feedback is delivered in coordination with a ‘reject’ judgment.  Without action, the feedback is just information (Ajjawi & Boud, 2017).  As one prominent author emphasised; feedback must be accessible, supportive and actionable otherwise it is like great art hung in a dark corner in need of illumination, to be seen and in need of being used (McArthur & Huxham, 2013).  Indeed, the persistent dissatisfaction in funding decisions made by peer review, demands a change in thinking about how we deliver decisions, as well as support future iterations and funding attempts.  This project, we hope, will be one important step towards developing a better, supportive and kinder peer review system. 


[1] A big shout out to my colleague, Dr Jan McArthur, who has played such a wonderful role in introducing me to the complexities and joys of feedback.

References

Ajjawi, R., & Boud, D. (2017). Researching feedback dialogue: An interactional analysis approach. Assessment & Evaluation in Higher Education, 42(2), 252-265. 

Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in higher education, 38(6), 698-712. 

Carless, D., & Boud, D. (2018). The development of student feedback literacy: enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315-1325. 

Derrick, G. (2020). Kindness under coronavirus. Nature, 581(7806), 107-108. 

McArthur, J., & Huxham, M. (2013). Feedback unbound. Reconceptualising feedback in higher education: Developing dialogue with students, 92-102. 

Merry, S., Price, M., Carless, D., & Taras, M. (Eds.). (2013). Reconceptualising feedback in higher education: Developing dialogue with students. Routledge. 

Photo credit: @helloimnik on Unsplash

Published by Helen Greaves

Helen is currently completing her PhD research which aims to assess the value of pond management for biodiversity conservation. Her work focuses on macroinvertebrate community assemblages and water chemistry analysis. Helen acts as secretary and as a co-organiser of the Norfolk Ponds Project meetings and sits as a student member on the European Pond Conservation Network’s steering committee.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: