For richer, for poorer

The main problem with the REF is the huge amounts of time and money that it consumes, distracting academics from scholarly activities, and diverting funds to employ squads of administrators to manage it. In a recent blogpost, I reported an analysis I did showing that the you could get pretty accurate prediction of the psychology RAE results by taking an H-index based on departmental addresses. In the comments on my blog, there is further evidence that this is also true for physics. (You can find the blogpost by googling 'alternative to REF2014'). Computing the H-index takes an individual 2-3 hours, as opposed to the 2-3 years for legions of people working on the REF. At present, we don't know whether such differences as exist between an H-index rating and the RAE panel ratings mean that the latter is better. For both psychology and physics, once you had taken H-index into account, additional variance in funding level could be explained by whether there were departmental representatives on the REF panel. This suggests that the additional value given by having expert assessment incorporated in the evaluation may just be adding subjective bias - which does not necessarily indicate any malpractice but could reflect the advantage that panel members have of intimate knowledge of how the panel works. Most of us don't like metrics, and I accept they may not work in the Humanities, but I would suggest that if we are not going to use a metric like this, then we need to do an analysis to justify the additional cost incurred by any alternative procedure. If the additional cost is very high, then we might decide that it is preferable to use a metric-based system, warts and all, and to divide the money saved between all institutions.