Uncommon Descent Serving The Intelligent Design Community

Why peer review is obsolete and what to do about it

Share
Facebook
Twitter
LinkedIn
Flipboard
Print
Email

In “Time to review peer review” (New Scientist 21 June 2012), astrophysics postdoc Andrew Pontzen argues,

I am not arguing to remove peer review entirely from the scientific publishing process but it does need a radical overhaul, and now is the right time to start. Peer-review offered a quality-control filter in an age where each printed page cost a significant amount of money. It’s not totally clear that the quality-control filter was ever particularly effective, but at least it gave journals a way to cut down the volume of print.

These days most physicists now download papers from arxiv.org, a site which hosts papers regardless of their peer-review status. We skim through the new additions to this site pretty much every day, making our own judgements or talking to our colleagues about whether each paper is any good. Peer-review selection isn’t a practical priority for a website like arxiv.org, because there is little cost associated with letting dross rot quietly in a forgotten corner of the site. Under a digital publication model, the real value that peer review could bring is expert opinion and debate; but at the moment, the opinion is hidden away or muddled up because we’re stuck with the old-fashioned filtration model.

He offers an alternative:

Imagine a future where the role of a journal editor is to curate a comment stream for each paper; where the content of the paper is the preserve of the authors, but is followed by short responses from named referees, opening a discussion which anyone can contribute to. Everyone could access one or more expert views on the new work; that’s a luxury currently only available to those inside privileged institutions.

If that happened with ID-friendly papers, the discussions would look very different. Oh wait, there would actually be discussions. That’s what’s different.

One problem with peer review is that it creates an opportunity to just keep unpopular ideas out of print, irrespective of their fact base. ID is hardly the only example. One can think easily of a variety of “controversial” topics that are probably difficult to address honestly.

Our own johnnyb offers another suggestion:

I think a good way to revise peer review is to remove the idea that the purpose is to provide quality control for the readership. That should be the editor’s job. The peer reviewer should provide quality control for the author.

In other words, what authors really need from the peer review process is a few people to read their paper carefully before publication. The editor isn’t necessarily an expert in the subfield. The goal of the peer reviewers should be for the benefit of the author – what is the reaction from a co-expert in the subject on the paper?

Even if the reviewer is wrong, it is good for the author to know what it is that people who read his paper will think, and to respond to it. Certainly there will be things that the editor wants to force the author to respond to, but the responsibility for that should be on the editor, not the reviewer. The editor should not in any way defer to the reviewer, only use them for informational purposes.

I think that rethinking the roles of the reviewer in this way would help most papers, and get paper authors *appreciating* reviewers rather than loathing them. Instead of reviewers being a group of people guarding the decision to publish or not, they are instead a captive audience for your ideas who help you shape them so that everyone understands them and their significance.

Thoughts?

See also: What gets past peer review these days? Immortality through exercise …

Comments

Leave a Reply