Metrics would ruin the REF

As long as scholars behave rationally, they will try to get their work maximum exposure by publishing it in widely read and highly cited journals. This creates a market value for journals, where those with a higher impact factor are more desirable, within any given field. That makes journal impact factors normalised per discipline a good proxy for REF output ratings. Yes, there is stochastic variation if we use them as a proxy: The quality of some articles will be overestimated, and the quality of some will be underestimated. But because we would be averaging over many publications per department, this would still be a consistent estimator. (It may be possible to factor this uncertainty into a confidence interval if one wanted to provide more than a point estimate of departmental quality.) I have seen some people argue that top journals are not the best journals. But I have not seen any convincing arguments. It's often people who haven't been able to publish in good journals who reject journal metrics. All that being said, things may be a bit more blurry (i.e., have a higher uncertainty) in the arts and humanities, compared to STEM, natural sciences, and social sciences, because people there tend to publish in books and because quality may be hard to assess objectively when it comes to things that include art or opinion as a component. But even that is not necessarily an argument against metrics because manual, qualitative panel assessments may be just as error-prone in assessing the quality of the work if it is subjective. With all that in mind, I have yet to see compelling evidence for the claim that journal impact would be a bad aggregate measure of output quality.

Best thing would be just to scrap the REF and TEF. Both a complete waste of public money. Give it to the health service.

A few thousand words of missing the point there. Metrics aren't very good at evaluating individuals, but they are good at evaluating *departments*, because individual errors average out. Given they are also cheaper and simpler than peer review, I don't understand why we are still having this argument. I can only assume there are a huge number of people who are perversely invested in the REF process.

There is little to no likelihood of consensus on any assessment scheme to replace REF and/or TEF and/or KEF. Many of us are frustrated at the amount time and opportunity cost associated with assessment ... particularly because this is diverted from the doing of core academic research, scholarship, learning & teaching, impact and knowledge exchange. It would be possible to model the extent to which metric driven approaches achieve similar outcomes to the current expert-led process and indeed, previous attempts to do this have been produced. IF the REF panel didn't anonymise the assessment process at the level of the individual outputs we could see at a much more granular level- what is happening. This would resolve the aggregated/disaggregated issues for units of assessment ... but of course, there is no incentive to do so for the REF panels ... and consequently everyone is guessing whether their individual output, with its individual citation/altmetric/etc. data was or was not well-correlated with the expert opinion delivered by the panel. I doubt our students would accept feedback which was only about the aggregate outcomes for their year group. Undertandably we are tasked with giving specific feedback and marks for their individual work. It is an odd juxtaposition that our academic community are given relatively high level, aggregate feedback and denied the chance to learn how their own individual outputs were rated so that they can understand which types of work, in which types of outlets will help develop their career.

What would happen if the REF/TEF were scrapped? Suddenly all academics would reduce the quality of their teaching and everyone starts doing shoddy research? Institutions like Cambridge/Oxford/Harvard are revered around the world, why? because of REF? If you know academia, you would know that it is a small world and your reputation travels before you. It is one profession where individual achievements are on public display for everyone to see and judge. What is the REF panel going to tell me that I can't read a paper or look at grant document and understand about its quality? Well infact REF panel wont even tell about any individual outcome. Just total nonsense ! All it does is keep the rent seekers and other businesses like exeternal reviewers for the internal assessments etc. thriving. Apparently being on the REF panel it self is very prestegious, yes so ahead create a market there as well and enable gatekeepers and well connected to ensure that those positions also go to a select few. Does It help people decide on the best universities? Given how dependent UK universities are on international stuednt income, get a grip - as one recruitment agent from the China commented - everyone knows there are 20 universites in the top 10 in the UK. After the REF everyone claims to be top of something! So wake up and smell the coffee. This business needs to be wound up pronto.