Uncommon Descent Serving The Intelligent Design Community

Robert J. Marks: Time to change the peer review system

Share
Facebook
Twitter
LinkedIn
Flipboard
Print
Email

Back to the way it usually was in Einstein’s day, says Robert J. Marks, of the Evolutionary Informatics Lab. Einstein’s only rejected paper was reviewed anonymously, as is the practice today:

The assumption that today’s peer-reviewed paper has been vetted by experts and therefore has been awarded a blue ribbon for excellence is far from the truth. Peer review often does not do its job. Consequently, today’s collection of scholarly literature is exploding in quantity and deteriorating in quality.

Peer review comes in different flavors. One is anonymous peer review, where the identity of the reviewers is kept secret from the author. Albert Einstein only had one anonymous peer review in his career — and the paper was rejected. This happened in 1936. A decade and a half earlier in 1905, Einstein’s annus mirabilis (remarkable year), he had published four breathtaking papers. One introduced the world to special relativity…

Robert J. Marks, “Einstein’s only rejected paper” at Mind Matters News

His “remarkable year” reviewers were themselves eventually Nobelists so they were actually peers. Marks again:

Much has changed. Peer review today is done largely by reviewers who are not peers. Professors often assign paper review to senior graduate students. Associate Editors often do not examine a paper carefully, defaulting to the recommendations of the reviewers. Editors typically parrot the opinions offered by the Associate Editor. More than once. I have been forced to offer tutorials, rich with references, to correct “peer” reviews made by technical simpletons. It’s a waste of time. A reviewer should know such things.

Robert J. Marks, “Einstein’s only rejected paper” at Mind Matters News

See also:

Further reading: Why is it so hard to reform peer review? Robert J. Marks: Reformers are battling numerical laws that govern how incentives work. Know your enemy!

and

Anti-plagiarism software goof: Paper rejected for repeat citations: The scholar was obliged by discipline rules to cite the flagged information repetitively.

Comments
ET - I'll send a paper on to an associate editor if I think it's interesting enough, and that doesn't require careful reading. With rejections, they are often clear, but I make sure I do do due diligence to check that my thinking is supported. But this level of care is much less than is needed for a proper review. There are usually lots of issues that can only be seen by specialists in the specific area of the paper, so I expect reviewers to read the manuscript with that level of care. My time is limited, so I have to ration it. This means that if I don't have to carefully read a submission to be able to reach a decision, I won't. If I'm not sure, I'll read more carefully.Bob O'H
May 18, 2020
May
05
May
18
18
2020
07:41 AM
7
07
41
AM
PDT
What kind of editor doesn't carefully read the papers they are approving or rejecting? Thanks to Bob for pointing out more problems with peer-review.ET
May 17, 2020
May
05
May
17
17
2020
07:45 AM
7
07
45
AM
PDT
Polistra @ 5 -
Journals with good editors are enough. A scientifically literate editor can always spot major problems with method or logic.
As a journal editor, I'd disagree. partly because there can be problems that needs a specialist to uncover, and also because we don't have the time to carefully read papers. I'd also acknowledge that editors do often spot problems. Another point is that most papers are improved by the review process - small issues can be corrected or clarified, and sometimes the whole paper can be changed to make it clearer.Bob O'H
May 17, 2020
May
05
May
17
17
2020
02:35 AM
2
02
35
AM
PDT
Adding: The major journals get BIG subscription prices on the basis that you're paying for selectivity. The editors palm off their selective work onto the peer reviewers who aren't paid. This is at least chintzy if not quite fraudulent.polistra
May 16, 2020
May
05
May
16
16
2020
09:56 AM
9
09
56
AM
PDT
The whole setup is unnecessary. Journals with good editors are enough. A scientifically literate editor can always spot major problems with method or logic. Details can be handled by correspondence or blog-style comments on the articles.polistra
May 16, 2020
May
05
May
16
16
2020
09:52 AM
9
09
52
AM
PDT
It is very telling that there aren't any peer-reviewed papers that support blind watchmaker evolution, ie unguided evolution/ evolution by means of blind and mindless processes. Why is that? Why is unguided evolution devoid of science and yet still considered to be scientific?ET
May 16, 2020
May
05
May
16
16
2020
08:01 AM
8
08
01
AM
PDT
T
You probably wouldn’t say it, but I will: would BIO-Complexity be a case in point?
Perish the thought. :)Ed George
May 16, 2020
May
05
May
16
16
2020
07:48 AM
7
07
48
AM
PDT
Ed George: "There has also been a proliferation of journals created in an attempt to provide credibility to a scientific view that is questionable, at best. These can most easily be identified by the fact that the authors of most of the papers published also belong to the editorial board of the journal." You probably wouldn't say it, but I will: would BIO-Complexity be a case in point?timothya
May 16, 2020
May
05
May
16
16
2020
02:18 AM
2
02
18
AM
PDT
A few decades ago the number of journals in any field was small, costly to produce and costly for libraries to purchase. As such, there was pressure placed on the journals to ensure that the papers it published were of high quality. The internet, however, has changed this. Today there are a large number of journals competing for papers. I have only published a couple papers in the last couple years but not a week goes by when I don’t receive an email from a journal asking me to submit a paper. There has also been a proliferation of journals created in an attempt to provide credibility to a scientific view that is questionable, at best. These can most easily be identified by the fact that the authors of most of the papers published also belong to the editorial board of the journal.Ed George
May 15, 2020
May
05
May
15
15
2020
08:24 PM
8
08
24
PM
PDT

Leave a Reply