Academics in Italy have boycotted assessment. What has it achieved?

As a coordinator of one of the sub-groups evaluating research in the present VQR I can assure the UK public that our evaluation procedure is alive and well. The participation rate has dropped only 3.3 % from the preceding VQR (95.3 to 92.0). There is a stable percentage of 5% of professors not publishing anything, so that 8% drops to 3% of real protesters. Source: document of the National agency for the evaluation of the universities, http://www.anvur.org/attachments/article/1005/vqr%20documento%20statistiche~.pdf I agree completely with the protest of the colleagues nationwide about salaries and funding, but I do not think they chose the right action. Having the (scarce) governmental founds distributed according to merit and not academic camarillas is a common interest. The present method has some shortcomings but nobody really wants to go back to the past.

As a protester to the VQR I would like to reassure the UK readers of three things: 1) the Italian VQR is a farce, it costs more than the amount of money the government gives for research and is based on a methodology that has been proved being just downright wrong; 2) the percentages given are provided by the agency organizing the VQR, so it's like asking the government of North Korea if there is a protest; 3) the source of the numbers notwithstanding, the numbers themselves count the papers collected, forcibly and without consent of the authors, by the agency; so anyone stating that it is a representative number is sorely mistaken; I won't suggest other reasons for such a mistake. The three above stated, I will add that yes, I am _VERY_ angry with the current establishment.

What is not highlighted in the above comment of colleague natali is that the distribution of the adherence to the protest is not uniform. There are many Universities in the country that have a percentage of submitted papers that is lower than 90% and one main and prestigious Institution where only the 86% of the papers have been presented. Let me discuss an example, in a different scenario. Let us consider a scientific paper submitted to a top-ranked journal, and assume that this paper has the aim of compare two different approaches, that we will name approach A and approach B. Now assume that the authors of the paper state that, for some reason, they was able to score the approach A in the 100% of the cases, while the approach B has been tested only in the 86% of the trials and, in the remaining 14% of cases the authors assumed a score of zero for the approach B. Now, as an Editor of the journal, what do you think about this paper ? Generally in these cases my suggestion is to reject the paper, not because of missing data, but because the paper is not technically sound. Now what I really cannot understand is why, in Italy, the common practice that we apply in the revision process of the scientific Literature becomes something of completely unknown when we have to measure the research quality of our institutions. Measuring the quality of the research of whole Departments, Institutions, of a Nation, wouldn’t adhere to much more rigorous scientific and methodological practices than the revision of a single paper ? In my opinion this is another clear example that the aim of what we call “valutazione della qualità della Ricerca VQR” (evaluation of the research quality) is not the improvement of the Research quality of our Nation but justifying an unequal and unfair distribution of the severe cuts that our Governments are applying to the public research funding. This is the thesis demonstrated, by the other things, in a recent book edited prof. Viesti and published by Donzelli. Therefore let me respond to my colleague that there is nothing to be happy of in this context. Its triumphalist tones are completely inappropriate. In Italy we should staring working very hard to completely rethink our evaluation system with the aim of promoting the improvement of the whole public research system.

First of all, I totally agree with previous 2 comments by migliardi and decaro. And, of course, with the ideas and comments of the authors of the paper, especially referring to technical (and political too!) aspects of the italian research assessment process. Further, in my opinion the present italian VQR is a kind of tool that, far from helping to improve the whole italian national academic and research system, it seems to have the only goal to depauperate and cut funds to most universities, especially in the Southern Italy, just to concentrate the (small) government funding to few institutions, generally located in the Northern Italy. "Strange" parameters, like for example the de facto use of the social and economic development level of the region where the University is (through the level of employment of the University graduates: obviously there is a better "chance" of employment where the productive development is high), are largely used to "punish" the less "lucky" Universities, heavily cutting their funds, and to reward (by just a "small" funds cut, anyway!) the Universities located in economically strong regions of the Country. The goal, I mean, seems to be not to improve and rebalance the quality of research and teaching in the whole Country, but "reward" few institutions to make them a kind of (many few!) centres of excellence. Another side of the coin is the "migration" of thousands of students from the Southern Italy regions to the Northern Italy Universities, with several effects: to improve, more, the Northern Italy regions economy, to increase the importance of the Northern Italy Universities, and to weaken, even more, the Southern Italy economy and its academic and research institutions, and so on, in a kind of "vicious circle". “Starving the beast”, the economy largely used term, seems to be appropriate in this case, where the “beast” should be the Southern Italy Universities, as well as their academics and researcher.