June 27, 2023, #44
In a Science Insider post titled ââQuietly revolutionaryâ plan would shake up the way U.K. universities are evaluatedâÂč, Cathleen OâGrady writes that âUnited Kingdomâs four national funding bodiesâÂč are changing âthe way the labor-intensive Research Excellence Framework (REF) exercise defines âexcellenceâ.âÂč Under these changes planned for the 2028 evaluation round âResearch culture will get more weight, with less emphasis on publications.âÂč
OâGrady writes that previous versions of the REF âhave placed too great an emphasis on the number of âoutputsââpeer-reviewed papers and other publicationsâÂč, effectively prioritizing publication numbers âover long-term projects or contributions such as software developmentâÂč The new approach would alter the weighting so that â25% of each universityâs score will rest on its assessment of âpeople, culture and environment,â up from 15% in the 2021 REF.âÂč Publication numbers are not an ideal measure of quality, but they do have the virtue of being relatively transparent and equally clear criteria could be hard to find.
One of the lead persons for this change appears to be âCatriona Firth, policy lead for research and culture at the national funding body Research EnglandâÂč, who âsays she and her colleagues will be looking for ârobust indicators and evidenceâ that universities can submit so the assessment âgoes beyond institutions making claims about how good they are.ââÂč Looking for robust indicators is good, but actually naming them may be harder. Unequal treatment of staff appears to be a problem with the current system. According to Firth the current system âcreated inequalities and low morale within departments, and incentivized universities to carefully time the end of short-term contracts to keep certain staff out of their REF submissions.âÂč
It is never easy to change an evaluation culture. Institutions that were successful in the old culture may be uncomfortable with the new criteria, especially if the criteria are vague. Nonetheless, this change could have positive effects, because many countries and institutions implicitly or explicitly use the REF as a model. Having the UK take the lead in moderating the pressure to publish may also reduce the number of dubious publications around the world.
1: Cathleen OâGrady, âQuietly Revolutionaryâ Plan Would Shake up the Way U.K. Universities Are Evaluatedâ, Science Insider, accessed 22 June 2023,https://www.science.org/content/article/quietly-revolutionary-plan-would-shake-way-u-k-universities-are-evaluated.