High REF scores linked to strong journal impact factors – study

When I served as a research dean we didn't bother doing any reading of articles. We weighted each article by the 5-yr JIF to come up with a collective score that we then ran simulations around to generate the best mix of articles to submit (based on not just the mean score but the variance of outcomes that could arise). Doing that predicted the REF outcomes with more than 85% accuracy. When I was involved in the Australian equivalent, our panel argued for doing away with individual reading of articles (saving hundreds of thousands of hours of people's time) because the JIF weighting model was very accurate in predicting collective scores (it is less accurate at the individual level but the who exercise is meant to be collective). Needless to say, this was not taken up and millions of dollars of productive time has been wasted since reading articles.

Most of these papers are written by AIs anyway, but don't worry, we have AIs to detect the papers written by AIs. The problem comes when the AIs start plagiarising the AI detection AIs. What we really need is an AI to detect AI plagiarism of AI. That'll sort it.

“in all fields, an article in a substantially above average citation impact journal has a reasonable chance of scoring 3* instead of 4*” - Right. How is this a good thing? I could understand if it said for example that it has a reasonable chance of scoring 3* instead of 2*.

In my discipline (history) I think part of the difference is that the high impact journals are often generalist, with a broader readership across the discipline (and beyond) rather than with a more limited chronological, thematic, or geographical focus of other journals. This is not to say that all articles in generalist journals are good or that good and excellent work isn't published in more specific journals. Rather I think often the difference in scoring and perception of significance lies in the 'framing' of the article in the introduction or opening and the claims it makes to address big/ important debates in one or more areas.