Uncommon Descent Serving The Intelligent Design Community

A novel suggestion at Nature: Publish the peer reviews

Denyse O'Leary
September 1, 2018
Culture, Intelligent Design, Peer review, Philosophy, Science
Share
Facebook
Twitter
LinkedIn
Flipboard
Print
arroba Email
The august peer review committee, unmasked/Louis Wain, The Bachelor Party, public domain

But can the internet handle all the spite and unseemliness?

Another risk is the ‘weaponization’ of reviewer reports. Opponents of certain types of research (for example, on genetically modified organisms, climate change and vaccines) could take critical remarks in peer reviews out of context or mischaracterize disagreements to undermine public trust in the paper, the field or science as a whole. Queries to eLife, The BMJ and EMBO Press about this problem revealed only one, mild example (see go.nature.com/2piygkb). But weaponization could be a greater concern for journals that publish work that is more likely to be politicized.

One precaution would be to add a disclaimer explaining the peer-review process and its role in scientific discussion. Opening up materials and establishing dialogues with journalists, politicians and the public is an opportunity to build trust and enhance understanding of the scientific process.

Published peer-review reports could also place editorial decisions under greater scrutiny and perhaps make editors more timid about overriding critical reviews (see go.nature.com/2bid8ag). Equally, published reports could boost appreciation for the role of editors in synthesizing and prioritizing diverse reviewer opinions… Jessica K. Polka, Robert Kiley, Boyana Konforti, Bodo Stern & Ronald D. Vale, “Publish peer reviews” at Nature

It’s not as if inherently political topics won’t be politicized anyway. Playing with a full deck limits the types of accusations that may legitimately be made.

See also: All peer reviews should be published, argues bioengineer

Comments
Bob O’H, I agree. But I think picking holes in the peer review process is a pointless excercise. The weaknesses of peer review have been known for a long time, but it is still the best way to disseminate merit based information. Certainly better than books and the internet. This being said, the biggest weakness in the science journal business is that there are very few papers published of research that completely inconclusive. I often wonder how many researchers have gone down dead-end paths simply because researchers that have previously gone down these paths have not published their inconclusive findings. R J Sawyer
If you want to judge the merits of the paper, the best option is to obtain the original data from which the paper is based.
A lot of journals require this already. But it's not enough, as the analyses need to be reproducible too (I think this is coming). I'm on the fence about publishing reviews, partly for the reason you suggest, and also because only reviews of accepted papers will be published. I also suspect that few reviews would be read. Bob O'H
Publishing the peer reviews sounds good in theory, but they can often be misleading. There is often two or three back and forths and edits to the paper before it is acceptable. Publishing the first part of the review will misrepresent the merit of the final paper. If you want to judge the merits of the paper, the best option is to obtain the original data from which the paper is based. R J Sawyer
The guide to arranging the deck chairs on the Titanic is too secret and anonymous. Open it up! Let everyone see the procedures needed to organize and fund the task force that determines the priority of canvas chairs vs leather chairs, the rank order of chairs nearest the rail vs chairs near the smokestack. Let us know who proofreads the Reserved cards for the celebrity chairs! Transparency solves all problems! polistra

Leave a Reply