Yes, peer review sucks. But attention-economy hellscapes would be worse

Obliging everyone to undertake post-publication review would aid discoverability in a world without traditional journals, says Robert de Vries

Published on
February 10, 2023
Last updated
February 10, 2023
Raised hands in a flaming landscape
Source: iStock

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please
or
to read this article.

Related articles

Reader's comments (7)

This might or might not "fit" quantitative sociology, but I have doubts that it applies or reflects the rest of sociology, let alone the humanities or most of the academy.
Thanks for your comment. As I said, I'm sure this is not a perfect system! But I would be interested to hear how you think it would not work for non quantitative sociology, or for the humanities or other disciplines. Could you expand on what you mean a little bit?
This might or might not "fit" quantitative sociology, but I have doubts that it applies or reflects the rest of sociology, let alone the humanities or most of the academy.
This might or might not "fit" quantitative sociology, but I have doubts that it applies or reflects the rest of sociology, let alone the humanities or most of the academy.
I fully acknowledge I may have misunderstood but surely such repositories have/will have a search function to enable discoverability. And in any case, discoverablity is a prerequisite of post publication review--isn't it? So, I am a bit confused as to how this proposal will work.
I think perhaps I should have offered a definition of 'discoverability'... An analogy with YouTube might help. YouTube has a search function, and if you know exactly what video you are looking for ahead of time, then you can easily find it. That is not what 'discoverability' means in this context. Instead think about what are the chance of coming across any particular video among the hundreds of thousands posted every day. Or to flip it around to the creator's perspective, what are the chances of anyone coming across the video you just posted amongst the flood - basically nill. I.e. despite being technically findable through the search function, your video still has very low 'discoverability'. In a laissez faire post-publication peer review system, papers would only attract reviews if people happen to read them and feel moved to leave a review. Hence if people were to simply upload their papers to a pre-print repository, the vast majority would never be seen or receive a review. This is a real issue for basically any online content creator or seller: YouTube video creators, Amazon sellers, people who sell video-games on Steam etc etc. To address this, creators/sellers have to engage in a whole host of strategies to try to get their creations/products seen by anyone - including things like hustling on social media (e.g. trying to get accounts with large followings to mention you) and trying the 'game' platform algorithms by manipulating keywords. This is the 'attention economy hellscape' I refer to in the article. I DO NOT want a future where scientists have to engage in these kinds of antics to get their papers seen. It would reward people who are good at this sort of thing (which is unlikely to be positively correlated with scientific competence) and people who already have large online followings. My proposed solution to this is twofold. First, separate the single firehose of new papers into discrete, topic-specific forums. And second, make sure that any new paper posted to a given forum is sent to at least one or two forum members, who will be required to give a review (or at least a thumbs up/thumbs down) on pain of losing their posting privileges. This is not something that e.g. YouTube could get away with, but these individual forums are supposed to be where scientists go to air and debate their ideas, so some kind of minimal engagement with other people's papers wouldn't seem to high a cost to bear.
The prestigious Journal of Truth is published by the learned Society for the Discovery of Truth. In a world where authors of research articles simply post their articles to preprint repositories, what can the society and the readers of their journal do to promote truth and expose falsehood? The obvious answer is for the Society to set up a reviewing organisation which would scour the internet for suitable articles and then get their trusted reviewers to review them. Then potential readers of articles which would, in the old days, have gone to the Journal of Truth for carefully vetted reading on Truth, would simply switch their allegiance to Truth Reviews. This would have several big advantages over the old system. There would be no publication and distribution costs for the Society to meet. They could also review papers which might not have been submitted to the Journal of Truth so their range could be far wider. And, perhaps most importantly, papers on Truth could be reviewed by reviewers from other disciplines – not peers but experts in other areas: the Relativist Review platform, for example, might review some of the papers on truth and perhaps encourage Truth seekers to see things from another perspective. So, yes, I agree peer review should die, and I think it would almost inevitably be replaced by a more flexible system of reviewing organisations. I have posted a more detailed account of this idea on the preprint platform arxiv.org.

Sponsored

Featured jobs

See all jobs
ADVERTISEMENT