The Scientist: “Is Peer Review Broken?”

_{William Dembski

February 10, 2006

Intelligent Design}

Share: Facebook; Twitter; LinkedIn; Flipboard; Print; Email

. . . Everyone, it seems, has a problem with peer review at top-tier journals. The recent discrediting of stem cell work by Woo-Suk Hwang at Seoul National University sparked media debates about the system’s failure to detect fraud. Authors, meanwhile, are lodging a range of complaints: Reviewers sabotage papers that compete with their own, strong papers are sent to sister journals to boost their profiles, and editors at commercial journals are too young and invariably make mistakes about which papers to reject or accept. Still, even senior scientists are reluctant to give specific examples of being shortchanged by peer review, worrying that the move could jeopardize their future publications. . . .

MORE

[See also Frank Tipler’s article on peer review in UNCOMMON DISSENT: INTELLECTUALS WHO FIND DARWINISM UNCONVINCING.]

Comments

Scordova That is all that I have ever presented in the only venue that counts - Journal publication. What I do on internet forums is mostly just for my own amusement and the pleasure it gives me to ridicule ideologues to my hearts content.John Davison_{February 14, 2006
February
02
Feb
14
14
2006
04:29 AM
4
04
29
AM
PDT}

The scientific method should be driven by empirical results, not by opinions.scordova_{February 13, 2006
February
02
Feb
13
13
2006
08:01 PM
8
08
01
PM
PDT}

In post #7 DaveScot asks how that has worked out for me. Why DaveScot, it has worked out very much to my satisfaction. I thought everybody knew that. So much so that I am contemplating a book "How to lose friends and and alienate people." I'll bet it will make the best seller list. "If you tell the truth, you can be certain, sooner or later, to be found out." Oscar Wilde "I have always felt that a politician* is to be judged by the animosities he excites among his opponents." Winston Chuchill * substitute obnoxious nasty old retired bench scientist. How do you like all them celery sticks with all that cheap Cheeze Whiz on them? Constipating aren't they? Good. That's the name of my game. Get out the Exlax.John Davison_{February 11, 2006
February
02
Feb
11
11
2006
03:05 AM
3
03
05
AM
PDT}

I don't give quizzes any more. I retired from academe in 2000 much to its relief.John Davison_{February 11, 2006
February
02
Feb
11
11
2006
01:05 AM
1
01
05
AM
PDT}

Frank Tipler's article on peer review is also in the ISCID archives. They share the same name, so I presume it is the same one that's in the book. http://www.iscid.org/papers/Tipler_PeerReview_070103.pdfcrandaddy_{February 10, 2006
February
02
Feb
10
10
2006
08:06 PM
8
08
06
PM
PDT}

Professor Davison wrote: "Anonymity still sucks. Got that? Write that down again." I prefer to remain annonymous, because it appears from your last two posts that you're planning to spring a pop quiz on us. Russ Riedigerruss_{February 10, 2006
February
02
Feb
10
10
2006
06:15 PM
6
06
15
PM
PDT}

The problems with signed reviews were discussed in the article Bill cited. It is a fact that the bulk of journal reviewing is not done by the most distinguished and senior scientists in a field, but by academics further down the food chain. People are only human, and it can be potentially damaging to a junior faculty member to criticize the paper of a senior scientist (or any scientist for that matter) in the reviewer's field. And I speak as someone who has reviewed over seventy papers and forty grants in the last six years. -DWDaveWatt_{February 10, 2006
February
02
Feb
10
10
2006
05:09 PM
5
05
09
PM
PDT}

"I prefer abusing my adversaries right up front." How's that worked out for you?DaveScot_{February 10, 2006
February
02
Feb
10
10
2006
04:23 PM
4
04
23
PM
PDT}

Just for the hell of it why don't you insist on identity here and see what happens? I'll bet you would lose quite a few blowhards and everyone else would suddenly become a little more cautious about what they said. Of course that will never happen. I don't know why I even made such a preposterous suggestion. I should know better. Bill Dembski lets everybody know who he is and so do I. What's the matter with the rest of the denizens of cyberspace? No guts no glory. Anonymity is nothing but license for cowardly abuse. I prefer abusing my adversaries right up front. It is great sport. That way they know exactly whom to loathe and why. War, God help me, I love it so! Me too George baby. Let's get it on!John Davison_{February 10, 2006
February
02
Feb
10
10
2006
04:08 PM
4
04
08
PM
PDT}

Sure and a lot of it is possible because of anonymity which was my only point. I sent a paper to Science once and the reviewer got so upset with it that he wrote to me to tell me what was wrong with it and thereby disclosed his identity. You know what I did? I wrote the Editor of Science, told him what the jerk had done, and demanded a new reviewer. My paper was subsequently accepted without a word changed. So much for anonymity. For a while I reviewed for Science but got so sick of seeing papers I had endorsed get turned down that I quit doing it. Anonymity is the cause for all of it. Anonymity still sucks. Got that? Write that down again.John Davison_{February 10, 2006
February
02
Feb
10
10
2006
03:57 PM
3
03
57
PM
PDT}

If you think peer review is broken in scientific publishing you should experience the process in the patent world where engineers publish or perish. I'm not all that familiar with the former but have been involved in at least a thousand patent reviews resulting in hundreds of grants. Patent examiners are simply lost in the increasingly esoteric world of computer technology. I swear to God I could patent a vacuum tube by changing the language to something that won't match old patents on vacuum tubes. The examiners are too young to recognize a vacuum tube couched in frilly new language so they search the patent database for prior art using the frilly keywords in the application. No hits? Must be an original idea! If something is common in the industry and never patented before because the first person to do it didn't get a patent, which is more common than not, the examiners have no clue about it and let that fly through too. There's supposed to be a test for obviousness that disqualifies a patent application - it can't be something that's obvious to an expert in the field - but since the patent office can't afford to pay for experts in esoteric or new fields there's really no disqualification anymore for obviousness. So you're left with no real requirement for novelty except novel language in the description and no real requirement for non-obviousness to experts. The only real requirement is a decent patent lawyer to make sure it's presented in the proper format. The name of a big company as the assignee doesn't hurt either. It's rubber stamp city. This really wouldn't be all that bad as the result is the opposite of what's happening in science where it's becoming more difficult to get published. It's getting easier to get a patent. The problem is a legal one in the commercial world. Once a patent is granted it's an uphill battle for infringers to prove the patent is bogus. Once granted the patent is valid until proven invalid and the benefit of doubt always goes to the patent holder because his invention was reviewed by the patent office and passed. So if I got that patent on a vacuum tube I could have my corporate lawyer send out ten thousand form letters to other corporate laywers asking for $100/year in royalties. The other corporate lawyer weighs the cost of paying $100/year to get me out of his hair or spend $100,000 on a legal battle he might not win. Some will call my bluff but a lot won't. There's a lot of abuse of the system like that. It's a racket and the only real winners are lawyers. Myself and a friend who were two of a dozen patent abstract reviewers at a $40 billion/yr computer company that shall remain nameless were scrupulous in trying to ensure that only really deserving, novel, non-obvious inventions were submitted to the patent office. We were often in a losing minority voting down the undeserving as there's a nasty numbers game that is played amongst the legal beagles of the big players - whoever has more patents, regardless of the quality of the patents, has the upper hand in negotiations. Cross license deals are made - you give me license on ten of yours and I'll give you license on ten of mine. Whoever runs out of patents to cross license first starts paying for cross licenses. Seldom are individual patent merits actually compared as the legal fees that come from court battles cost more than most of the individual patents are worth! I hope I don't sound too cynical but it's a valid cynicism. I can only imagine that similar vested interests, politics, and legal shenanigans pollute the integrity of the academic publishing world too.DaveScot_{February 10, 2006
February
02
Feb
10
10
2006
03:34 PM
3
03
34
PM
PDT}

It is the editors that are the culprits. They can determine the fate of a paper by simply choosing which of their reviewers they want to read it. Also the anonymity of reviewers must be abolished. It is a disgrace. They would be a damn sight more careful if they had to attach their name to what they write. Also all the reviewer comments should be revealed to the author so he can respond to the reviewers personally. That goes for internet forums as well in spades. Anonymity is nothing but licence for ideological bigotry wherever it is permitted. I should have refused it at my blog but I had no means to enforce it or I would have. Anonymity in any form sucks. Got that? Write that down.John Davison_{February 10, 2006
February
02
Feb
10
10
2006
03:20 PM
3
03
20
PM
PDT}

Peer review - of both manuscripts and grant applications - has been under fire for decades. No one has come up with a better system to replace it - including any of the people mentioned in the Scientist article. -DaveWDaveWatt_{February 10, 2006
February
02
Feb
10
10
2006
03:05 PM
3
03
05
PM
PDT}

Maybe it's a reflection of the declining number of scientists who believe that bearing false witness is a sin that will be punished whether or not they're caught at it by their peers? Getting published hasn't any absolute moral restrictions. It's a Darwinian jungle out there where the rule is every scientist for himself. Just a guess...DaveScot_{February 10, 2006
February
02
Feb
10
10
2006
02:31 PM
2
02
31
PM
PDT}

You must be logged in to post a comment.

Leave a Reply