Further to “Retract that, sir, or face the consequences! Er, maybe”: When you think of it, peer review is a form of crowdsourcing. Asking a crowd of experts can lead to more accurate knowledge or to strengthened conviction and a sense of righteousness and harmony with reality without more accurate knowledge.
In short, it can sometimes lead to disaster.
In “Peer Review; Last Refuge of the (Uninformed) Troll,” David M. Hoffer recalls grate moments in peer review:
In the fifth century BC, Empedocles theorized that one could see by virtue of rays emanating from one’s eyes. Falsifying this notion required no more than pointing out that one cannot see in a dark room. Despite this simple observation, his theory enjoyed substantial support for the next 1600 years.
Galileo died while under house arrest for supporting the notion that the earth orbited the sun. His was convicted in part on the basis of peer reviewed literature of the time insisting that the movement of the planets as observed from the earth could be explained by the planets simply reversing direction in orbit from time to time.
For nearly two thousand years, into the early 1800’s, when people fell ill, the peer reviewed literature confirmed that the best course of action was to let some blood out of them. The simple observation that death rates increased when this treatment was applied was dismissed out of hand on the premise that, if it was true, it would appear in medical journals. Sound familiar?
History is replete with examples of what seems today to be utterly absurd ideas. Ideas which stubbornly refuse to die, sustained in part by the equally absurd notion that evidence to the contrary was not to be accepted simply because it hadn’t appeared in the “right” publications.
Guy’s talking about climate science, and the point stands, no matter What’s Up With That ( 😉 ): Once we make something other than evidence the seal of good science, we set ourselves up for expensive flimflam. Self-righteous incompetence. And important nonsense. To say nothing of persecution of people who unwisely insist on putting evidence first.
Or worse. Much worse. Legions of unnecessarily dead people would (but can’t) testify to the importance of determining whether medical treatments and policies in particular are likely to actually work before testing them in life and death situations.